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ABSTRACT 


Three discrete reliability growth models using fractional failure reduction, referred 
to as failure discounting, were developed to estimate changing system reliability. Each 
of the models is designed for use when testing is performed until a fixed number of 
failures have been observed and attribute data, success or failure, is available for each 
trial. The first reliability growth model applies failure discounting to the maximum 
likelihood estimate for a proportion. The second and third models use a modification of 
an exponential reliability estimate employing linear regression and a weighted average 
technique respectively along with failure discounting to track changing reliability. Two 
failure discounting methods were used with each reliability growth model. The first 
method reduces past failures by a fixed fraction at a fixed interval. The second method 
uses the upper confidence bound for the reoccurrence of each failure cause as the 
discounted failure value. The performance of the reliability growth models with 


varying reliability growth patterns was evaluated with a Monte-Carlo simulation. 


} THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may 
meemm@ave been exercised for all cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and 
logic errors, they cannot be considered validated. Any application of these programs 


without additional verification is at the nsk of the user. 
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I. INTRODUCTION 


Accurately determining the reliability of a newly designed piece of hardware is an 
important part of material acquisition in both the mulitary and civilian sectors. 
Significant amounts of both time and money are spent testing new systems to 
determine if they meet specified reliability requirements. Since most organizations are 
under constant pressure to conserve both time and funds, it 1s important to develop 
methods of estimating system reliability at minimum cost. When conventional 
estimators for system reliabilitv are used, verifying high reliability requires a great deal 
of test data costing both time and monev. This problem is compounded bv 
technological advances which produce more complex and highly reliable hardware at 
an ever increasing cost. Developing new and more accurate methods of estimating 
reliabilitv that reduce required testing could result in significant savings of resources. 

Early versions of newly designed equipment often exhibit low reliability. As 
engineering designs are tested and improved the system reliability usually grows 
rapidly. This process continues throughout development phases and into field use. By 
the time a product design reaches its final configuration, a large amount of test data 1s 
usually available from tests performed on previous configurations, subcomponents and 
parts. Since conventional estimators assume a constant reliability, it is necessary to 
wait until the finished piece of hardware is available to begin reliability verification. 
This practice is not an efficient use of constrained resources because results from 
previous testing are discarded. If all the data available could be used to estimate final 
configuration reliabilitv, the lead time and cost of the finished product could be 
reduced. Reliability growth models serve this purpose. They use data from testing at 
all phases of development to establish a reliability growth pattern. This growth pattern 
can then be used to provide current estimates of svstem reliability with less testing than 
standard testing programs. . 

Reliabilitv growth models can be classified into two categories, discrete and 
continuous models. This paper will address discrete reliabilitv growth models that 
employ failure discounting. These growth models are used to determine reliability for 
svstems Where test results can onlv be stated in terms of success or failure, ie; attribute 


data. Failure discounting allows the user to fractionally discount past failures when 
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resultant corrections to equipment are verified through subsequent testing, as actual 
corrections to the cause of failure. The use of failure discounting makes it possible for 
previously linear estimators to track a non-linear growth pattern. 

The objective of this paper is to develop and evaluate the performance of discrete 
reliability growth models using failure discounting. Three different reliability growth 
models are proposed and evaluated with two different methods for discounting past 
failures. Monte-Carlo simulation is used to evaluate each of the three growth models 
under both discounting methods. The simulation uses actual reliabilities to generate a 
random growth pattern. From this growth pattern, test data is then generated for the 
known system reliability at each testing phase. Each of the reliability growth models 
can then be applied to the test data producing an estimated growth pattern. The 
performance of each reliability growth model is evaluated by comparing the estimated 
growth pattern with the actual reliability in each phase. | 

The following chapter will address each of the three reliability growth models 
used and the two methods of failure discounting emploved. Chapter III contains a 
discussion of the computer simulation used to evaluate each of the models. Chapter IV 
presents the results of the evaluation and Chapter V summarizes the study stating 


conclusions and recomendations for future study. 


et 


Il. DISCRETE RELIABILITY GROWTH MODELS 


A. BACKGROUND 

Three different reliability growth models and two different techniques for 
fractionally discounting past failures are discussed in this chapter. The reliability 
growth models can be divided into two categories. Models using a modification of the 
maximum Itkelihood estimate, MLE, for a proportion, referred to as the MLE With 
Discounting Model, a model referred to as the Weighted Average Model and an 
Exponential Regression Model. All of the models may be applied with or without 
failure discounting and are independent of the discounting method. 

Two different methods for fractionally discounting past failures are also 
discussed. The first method is referred to as the Standard or -Straight Percent 
Discounting Method. The second was proposed by David K. Lloyd and will be 
referred to as the Lloyd Method. Before the specific details of each growth model and 
discounting method are discussed, several terms need clarification. 

Reliability, R, as referred to in this study is defined as the probability ofa system 
successfully completing a single trial or operation cycle, Equation 2.1 . R will be used 
to denote an estimate of the actual reliability R. 


R = Pr( SINGLE TRIAL SUCCESS ) (eqn 2.1) 


Discrete reliability growth models are normally used for trials which can only be 
evaluated in terms of a binary variable, success or failure, i.e. attribute data. Exactly 
what constitutes a trial is normally defined by the test agency but could include, for 
example, a single rocket engine ignition attempt or a single test firing of a missile up to 
and including target .destruction. To use the reliability growth models discussed here, 
trials must be defined such that they are discrete and can be evaluated as a success or a 
failure. 

A testing phase is defined as a collection of trials, one or more, for which the 
configuration of the system being tested is unchanged. This implies that the actual 
system reliability remains constant throughout a single phase. If a _ test-fix-test 


methodology is employed, design changes or “fixes” are implemented after each failure. 


|e 


In this case, a phase will consist of one failed trial and all the successful trials leading 
up to the single failure. If on the other hand a test-find-test methodology is used, a 
single phase may include numerous failures. Under both methodologies, a phase is the 
collection of all trials between configuration changes. [Ref. 1] 

Because the underlving reliability of the system may be different from phase to 
phase, a new estimate of reliability must be computed for each and every phase. As 
such, R, ,denoting system reliability and #, denoting the estimate of system reliability 
in phase k, will be used throughout the paper. 

Annotating the cause of each failure is critical to all discounting methods 
considered in this study. How the failure cause is defined, is dependent upon the 
judgement of the model’s user and the type of system to which the model is applied. If 
the model is used to determine the reliability of a single component, failure cause refers 
to the reason for failure such as a broken cam-shaft or fuel stoppage. If the model is 
applied to determine the reliability of a system of components, the failure cause may 
refer to the failure of a sub-component or the specific reason for failure. To apply the 
discounting methods discussed in the paper, the model user must determine if two 
observed failures are due to the same cause. It will not be necessary, however, to 
enumerate all possible failure causes. The level of detail at which failure causes are 
defined is up to the user and should be determined before any testing is begun. The 


discounting methods studied will be discussed in the next section. 


B. DISCOUNTING METHODOLOGY 

Weaknesses in a system design become known as they cause failures during 
testing. As weaknesses are identified and corrective “fixes” are implemented the 
probability of a repeat failure due to the corrected cause should be reduced and svstem 
reliability improved. This concept forms the basis for failure discounting. Once a 
system weakness has been corrected and the improvement validated through further 
testing, fractionally discounting the past failure is a method of reflecting the improved 
system reliability in the previously collected data. When using standard reliability 


estimators such as the maximum likelihood estimate, Equation 2.2 


Number of Successful Trials 
——————————— SS (eqn 2.2) 
Total Number of Trials 
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only the data from the phase of interest can be used to determine system reliability 
because system reliability is different from phase to phase. If the proper discounting 
methodology can be developed, failure data from all past phases could be discounted 
and made compatible with the data from the current phase. Increasing the amount of 
data available for computing the estimate of current system reliability will increase the 
accuracy of the estimate and reduce the amount of testing necessary to verify system 
reliability. 

Of equal importance to a method for discounting past failures is a penalty system 
that will restore all or some portion of a previously discounted failure if its failure 
cause should reoccur. Ifa previously implemented “fix” proves not to have corrected a 
design weakness, the discounts applied to its failures will diminished - erased reversing: 
the effect of discounting. A properly designed penalty factor will make the discounting 
method self-correcting. 

In order for a discounting method to be useful, it must be flexible enough to 
handle many varied reliability growth patterns. To achieve this flexibility, discounting 
methods with one or more input parameters are proposed. Regardless which 
discounting method is used, experience and good engineering judgement will be 
necessary to choose input parameters that are compatible with both the reliability 
estimator used and the proposed test plan. If improper parameters are applied, the 
model may consistently over or underestimate system reliability. The following 
sections describe the specific steps for applying two failure discounting methods 
including examples of their use. 

1. Straight Percent Discounting Method 

The Straight Percent Discounting Method removes a fixed fraction of a failure 
each time a predetermined number of trials are observed without reoccurrence of the 
failure cause. Before a failure is discounted, the design “fix” correcting the failure cause 
Should be verified by further testing. This concept forms the basis for the discounting 
method. The fraction of a failure removed, F, is referred to as the discount fraction. 
After the initial occurrence of a particular failure cause, anv trial in which the failure 
cause is not repeated is considered a success. The number of successful trials between 
application of the discount fraction is referred to as the discount interval, N. Both N 
and F are input parameters specified by the user. They remain constant throughout 
failure discounting computations. To use the Straight Percent Discounting Method, it 


is necessary to record the number of successful trials since the last occurrence of each 
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observed failure cause, referred to as M. Failure discounting is performed at the end of 
each testing phase using the values for N, F and M. 
The mathematical representation of the straight percent discounting method is 


shown in Equation 2.3 


ADJUSTED FAILURE = (1 — F)ENT(CM/N) (eqn 2.3) 
F = Fraction of a Failure to be Removed 
M. = Number of Successful Trials Since Last Failure for the Failure Cause 


N 


The Discount Interval Expressed in the Number of Successful Trials 


INT is a function which returns the truncated integer portion of its argument. M 1s set 
to zero at each occurrence or reoccurrence of a particular failure cause. This causes the 
value for adjusted failure to be set at 1.0 erasing anv previously applied discounting. 
This also functions as a penalty factor allowing the model to correct itself should past 
discounting prove unwarranted. By altering the choice of N and F the amount of 
discounting applied can be varied from none to total loss of a failure. This allows the 
method to be applied in situations verying from little or no reliability growth to rapid 
reliability growth. 

An example will help clarify the use of the method. Consider the test results 
included in Table 1. The data displayed in the table is from a test-fix-test program in 
time order sequence. A design fix was applied to the equipment after each failure. As 
such, the actual system reliability does not remain constant. The outcome of each trial 
was generated from the geometric distribution using the actual reliability listed in the 
table. The cause for each failure was generated using a stochastic process and has 
been coded using letters. No more information concerning failure cause is required 
because the data are used to determine if the current failure cause has occurred 
previously. 

To. apply Straight Percent Discounting the two input parameters must be 
determined. For the example 

N = 3 

F = .25 
will be used. The Straight Percent Discounting Method is applied sequentially, failure 
by failure, recomputing M, the number of successful trials since last failure, for each 


failure cause as the next data entry is included. At the end of each testing phase, the 
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TABLE 1 
SAMPLE TEST RESULTS 


TRIAL FAILURE “PAILURE CAUSE 


NUMBER NUMBER A B C D RELIABILITY 
Ss s s § 358 
2 1 F § § § 358 
3 2 S F Ss § 12 
4 3 S S F S§S 518 
5 4 S S$ S F 596 
6 Sas ss 627 
7 5 S S F §S 627 
8 | aa om 
9 > See om) 
10 6 S F Ss § 721 
ll So esas, S 780 
12 S © 88 780 
13 Ss s Ss § 780 
14 Ss s Ss § 780 
15 Ss s s § 780 
16 SS 780 
17 Ss s Ss S§ 780 
18 7 F S§ S § 780 


S: DENOTES SUCCESS 
FP: DENOTES FAILE RE 


value of M, along with the input parameters F and N, is used in Equation 2.3 to 
determine the adjusted failure value. This procedure is applied to the data in Table 1. 
When the sixth failure is included, all failures due to Causes A, C or D have 
values for M greater than or equal to three causing the discount fraction to be applied. 
Computation of the adjusted failure values for Cause A is illustrated in the following 


calculations. 
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ADJUSTED FAILURE = (1 — .25)!N1(8/3) 
“iD 
5625 


Failure two and failure six were both due to Cause B which resets their corresponding 
M values to zero and their adjusted failure value to unity. The results for all failures to 


date are shown below. 


FAILURE Peleenee AIUS TED 
NUMBER M CAUSE FAILURE 


l 8 CHU SISITEN 9625 
2 0 CAUSE B 1.0000 
3 3 CAUSE C 7500 
+ 5 CAUSE D 500 
5 3 GAESE C .7500 
6 0 CAUSE B 1.0000 


Finally, failure seven is included and the failure discounting computations are 
completed. The number of trials, eight, causes further discounting for all previous 
failures except for the first failure which is also due to Cause A. The final results of 
applying the straight percent discount method to the data in Table 1 appears in Table 
2. The values in the adjusted failure column are the current values of each fractional 
failure after trial number eighteen has been included. For example, the two adjusted 
failure values due to Cause B have recorded eight successfull trials since the last failure 
due to Cause B and INT (8/3) = 2. Therefore, the adjusted failure value equals 
(.75)* = .5625. The remaining adjusted failures are computed in the same manner 
yielding the fractional failure values after-eighteen trials and seven failures. 

2. Lloyd Failure Discounting Method 

The underlying premise of the Lloyd Discounting Method states that the 
fraction of a failure removed by the discounting method should not be arbitrarily 
chosen. Some statistical basis should be used to determine how much, if any, a past 
failure should be discounted. Lloyd proposes using the upper confidence limit for the 
probability that a failure cause will reoccur as the discounted value of its corresponding 


failure. This technique allows the user to control the amount failures are discounted by 
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TABLE 2 


STRAIGHT PERCENT DISCOUNTING 
OF EXAMPLE DATA ( TABLE 1 ) 


TRIAL FAILURE FAILURE CAUSE) AbIGsrE) 


NUMBER NUMBER A B C D FAILURE 

1 Sse ss 
2 i F § § § 1.0000 
3 2 See SG 5625 
4 3 Ss S.F_S 4219 
5 4 Se So Se 3164 
6 SS SS 
7 5 Sise5__S .4219 
8 Sse S ss 
9 Ss Ss Ss § 

10 6 S F §s § 5625 

11 §S s Ss § 

12 S gSiee Ses 

13 S? 1S. esas 

14 S aie 5 

15 SS Sass 

16 S s Ss § 

17 Ss s Ss § 

18 7 F § S § 1.0000 


§: DENOTES SUG@ESS 
Fi. DEN@MES EAI imine 


specifving the level of confidence for the confidence bound. As such, a single input 


parameter, the confidence level y, is required to perform failure discounting. {Ref. 2] 
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The discounting equation is derived from the upper confidence limit for the 
probability of failure due to a particular failure cause. This confidence interval is based 
on the number of successful trials recorded since implementation of the “fix” for the 
failure cause. Assuming the “fix” is implemented immediately after failure, this will be 


equivalent to the number of successful trials since last failure, previously defined as M. 


Ce Oo Diet OR PAILURE IN A SINGLE TRIAL 
= Sos 
eae TONAL CONPIDENCE LEVEL 
Pipe bEheOr Sb GCEoSbuL TRIALS SINCE LAST FAILURE 
Et (y) Se he GeO MDE NC ral T MIT GOR RELIABILITY 
a — ye 
Bi = 
Y= Pr{R2 Zp] 
Pr | a L~ Bry) | 
Pr[Q S$ 1-Byy)] 


MPPER CONFIDENCE LIMIT forQ =1—-(1-—7)!/M (eqn 2.4) 


ADJUSTED FAILURE =1-(1-y)'/M 9 formM>0 — (eqn2.5) 
= 1.0 for M = 0 . 


The failure discounting equation is derived in Equations 2.4 and 2.5. Because the 
derived equation for adjusted failures is not defined for M equal zero, the adjusted . 
failure value is set equal to one where M equals zero. This is the logical choice since no 
discounting should be applied to a failure cause that is the current cause of system 
failure. This also has the advantage of providing a penalty factor that will restore 
fractional failures to unity if a failure cause reoccurs. 

To apply the Lloyd discounting method, the confidence level. y, must be 
specified. The method is then applied sequentially, failure by failure, recomputing M, 
the number of successful trials since last failure, for each observed failure cause as the 
data from each trial is included. At each step, M is used in Equation 2.5 to compute a 


new adjusted failure value for each observed failure. 


An example will help clanfy the use of the Lloyd method. Applying the 
method to the sample data in Table 1 with y equal to .99 yields the following results 


after the second failure is included. 


FAILURE FAILURE ADJUSTED 
NUMBER M CAUSE SOMO 


l l CAUSE 2 9900 
2 0 CAUSE B 1.0000 


Including the data for the third failure, due to failure Cause C, both Cause A and B are 


further discounted. The adjusted failure value for Cause A is computed below. 


M = 2 
ADJUSTED FAILURE = 1 - (1 - .99)!/2 
=1-/.01 
=1- 1 
ane 


The complete results are shown below. 


FAILURE FAILURE ADJUSTED 
NUMBER M CAUSE FAILURE 


I 2 CAUSE A 9000 
2 l CASE 5 722200 
3 0 CAUSE © 1.0000 


The process continues sequentially, adding the next failure, recomputing M, and 
computing the adjusted failure values for each previous failure. The final results after 
all seven failures are included is shown in Table 3. Note, as a failure cause reoccurs its 
corresponding value for M is set to zero forcing the adjusted failure value to unity. 
This concludes the description of the two discounting methods covered in the 


study. The following section will describe the three discrete reliability growth models. 
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TABLE 3 


LLOYD DISCOUNTING 
OF EXAMPLE DATA ( TABLE I ) 


TRIAL Pelle realURE CAUSE ADJUSTED 
NUMBER NUMBER eo. Ceo FAILURE 


SEDO E NOLESSUCCESS 
bor VOlees PAIL RE 


l SS Ss 
2 i F § § § 1.0000 
3 2 ae E77 
4 3 S$ S§ F § 3421 
5 4 SOS a 2983 
6 S$ s s § 
7 5 3 6 oes 3421 | 
g Ss womes 
9 S. Saeicmes 
10 6 Seas «6S 4377 
1! Cit SieeeS. 0S 
12 S$ s s § | 
13 S s s § 
14 S$ s s§ § 
15 S$ §s s § 
16 cess 6S 
ig S§S §s § § 
18 7 F S§S S § 1.0000 


C. RELIABILITY GROWTH MODELS DESCRIPTION 
Of particular interest in this study are test plans which terminate each test phase 
with system failure. Such test plans are known as testing to a fixed number of failures. 


Only one of the three reliability growth models proposed can be used for test scenarios 


zl 


which terminate testing after a fixed number of trials. Because of this limitation, only 
test plans which test to a fixed number of failures are considered in this study. 

Three discrete reliability growth models are presented in this section, each of 
which uses a different technique to track changing system reliability. The first model, 
Viaximum Likelihood Estimate With Discounting, is derived from the conventional 
maximum likelihood estimate of reliability. Failure discounting gives the model the 
ability to track changing reliability. Two additional models derived from an 
exponential single phase reliability estimate referred to as the Weighted Average Model 
and the Exponential Regression Model are also discussed. The MLE With 
Discounting model will be presented in the next Section. 

1. Maximum Likelihood Estimate With Failure Discounting 

The most commonly used estimate of system reliability is the maximum 


likelihood estimate Equation 2.2 restated below. 


Number of Successful Trials 
Se _ (eqn 2.2) 
Total Number of Trials 


Where the number of successful trials equals the total number of trials minus the 


number of observed failures. Because all testing considered is conducted to a fixed 


k 
NO. SUCCESSFUL TRIALS = E (T;- 1) (eqn 2.6) 
1= 


number of failures, the number of successful trials simplifies to Equation 2.6. Where T. 
equals the number of trials following the (i-— 1) ** failure including the 1 th failure and 
k equals the total number of failures in the test program to date. 

Two problems arise when using Equation 2.2 to estimate reliability growth. 
First, the estimate takes a large number of trials to accurately estimate system- 
reliability. Second, the estimator requires a constant reliability, R, for each trial. 


Because the reliability at each phase, R,, may not be constant with k; only the test 


k? 
data from the phase of interest may be used to estimate reliability. Failure discounting 
is introduced into the model in an attempt to adjust the underlying reliability for data 
from previous phases, making it compatible with the data from the current phase. This 


mcreases the amount of data available for estimating reliability at each phase, 


yi 


improving the accuracy of the estimate. Choosing the correct discounting method with 
the correct parameters is critical to this model because the previous test data must be 
properly adjusted to reflect an underlying reliability that 1s compatible with the current 
phase. This problem is difficult to solve because the actual reliability in each phase 1s 
unknown. 

Introducing failure discounting creates a computation problem because the 
number of failures is no longer an integer value. This problem can be resolved in one 
of two ways. The failure can remain a fractional value or the failure value can be reset 
to unity by adjusting the observed number of trials up to and including failure. Which 
ever method is chosen, the ratio of failures to total trials must remain constant or the 
estimate of reliability will be altered. Returning the failure value to unity and adjusting 
the number of trials is the method chosen to resolve this problem. This solution allows 
the data to retain its original structure of the number of trials up to and including 
failure. 

Adjusting fractional failures and the number of trials up to and including 
failure is accomplished by dividing both values by the fractional failure value. This 
returns the number of failures to unity and adjusts the number of trials such that the 
ratio between the number of failures and the number of trials remains constant. The 
adjusted number of trials may not be an integer value. This does not present a problem 
for computing the MLE. In cases where the total number of trials must also be an 
integer value, the adjusted number of trials is rounded to the nearest integer. This will 
have a minimal effect on the ratio of failures to trials. 

For example the following data is the result of applying a failure discounting 
method to observed test data. 

FRACTIONAL FAILURE. = .5 
NUMBER OF TRIALS = 6 


Dividing both values by .5 adjusts the fractional failure to unity and yields an adjusted 


failure value. | 
FAILURE VALUE = 1.0 
ADJUSTED NUMBER OF TRIALS = 12 
The ratio of failures to trials remains constant. 
0/6 = 1/12 = .0833 


Zs 


In general, Equation 2.7 is used to adjust the number of trials when the adjusted failure 


OBSERVED # of TRIALS 
ADJUSTED # TRIALS = ———_——————_Ss (eqn 2.7) 
ADJUSTED FAILURE 


value is returned to unity. 

The reliability growth model is applied to the data sequentially, phase by 
phase. After the test data from a phase is recorded, the discounting routine is applied 
to the data and an adjusted failure value is computed for each observed failure. The 


adjusted number of trials for each observed failure is computed using Equation 2.7. 


_ () ADJUSTED TRIALS) — J 
ss ) ADJUSTED TRIALS 


iy 


(eqn 2.8) 


ae 
ll 


TOTAL NUMBER OF FAILURES IN THE TEST PROGRAM TO DATE 


Finally, the estimate of system reliability in phase k is computed using Equation 2.8. 
This procedure is repeated after each testing phase is completed yielding an estimate of 
system reliability for each phase. These estimates can then be used to plot the 
estimated reliability growth pattern. | 
To illustrate the use of the model, the discounted failure values from Table 2 
are used to compute an estimate of system reliability. Because each data point 
represents a development phase, the estimate will be of phase seven reliability. The 
results of the computations are shown in Table 4. 
Note, the sum of the adjusted trials is not rounded to an integer value. Using 
only the data in the seventh phase, the MLE without discounting is 
jie 1) ews) 
The MLE using all test data without discounting will yield 
fe ES ee Olle . 
For this example, the MLE with discounting provides an improved estimate of actual 
reliability; i.e. R = .744 is closer to the actual reliability value, .78. 
When testing is terminated at a fixed number of failures, the MLE has an 
additional shortcoming; it is conservatively biased. Breaking the data into sets of trials 


up to and including a single failure, the number of successful trials in a set always 
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TABLE 4 


MLE WITH DISCOUNTING 
OF SAMPLE TEST DATA (TABLE 2) 


PHASE # CAUSE ADJ. FAILURE ADJ. TRIALS 
| A 1.0000 2.0000 
2 B 5625 1.7778 
3 E 4219 Dee 
4 D 3164 3.1606 
5 é 4219 4.7405 
6 B 5625 5.3333 
7 A 1.0000 3.0000 
i= 7 ¥, = 27.3824 
fle 7 | 
R, = ( 27.3824 — 7) # 27.3824 
= .74436 
R = .78 


equals the number of trials in a set minus one. As such, the MLE, Equation 2.2, mav 
be stated as follows for each set of data. 





> ote | 
R = 
xX 
Pee NC MBER OF TRIALS UP TOAND INCLUDING FAILURE 


The expected value of R 1s derived in Equation 2.9. Table 5 shows the E[R] compared 
to the actual reliability R. Clearly, the maximum likelihood estimate will be very 
conservatively biased for low reliabilities. (Ref. 3: p.34] 


Zs 


Biz) = sR a 
=1- E[1/X] 
E[1/X] = (1—-R) + (R/2)(1—-R) + (R2/3)1—-R) + ... 
= 1— R+ (R/2) — (R2/2) + (R2/3) — (R3/3) + «. 
Inl-R) = —R —(R*/2) —(R3/3) —-... 


~(1/R)In(l-R) = 1 + R/2 + R23 +... 
“EL 1/X] = In(1 — R) — (1/R)InQ — R) 


E[R] = 1 + (1/R)In(1 — R) — In(1 — R) (eqn 2.9) ; 


S¥0ts bom) 
MUE EXPECKED VALUE 


R EL R| 
! 052 
3 168 
5 307 : 
ii 484 
8 598 7 
: 9 744 ; 
99 953 
999 993 


This bias can be offset by the proper choice of a discounting parameter; but, 
the development of a less biased estimator would be advantageous. The Exponential — 
Single Phase Estimate, introduced in the next section, demonstrates a smaller { 


conservative bias than the MLE when applied to geometric data. 
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2. Exponential Single Phase Reliability Estimate 
The Exponential Single Phase Reliability Estimate was developed for the 
special case of testing until the first failure 1s encountered. Test data of this form has a 
geometric distribution with the parameter, R, system reliability. As shown in the 
previous section, the maximum likelihood estimate is conservatively biased when 
applied to geometric data. In an attempt to reduce this bias, an estimator of the 


following form was suggested. 


To complete the development of the estimator, an unbiased estimate, A, 
for A was sought such that 
E[A] = —-In(l — R)=-A 
The estimate is derived as follows. Let X, equal the number of trials up to and 


th set of geometric data and R. equal the system reliability for 


including failure for the i 
the i '® set of data where i = 1, 2, 3, ..., mumber of failures. Assume that 
Y, = f( X, ) is an unbiased estimate of A. Then 

oO | 

ac.) RX | (1-R) = —in(l — R) 

X= | 

=-R+R*+R°+.... 

Solving for f( x, ) by equating coefficients of terms with like powers yields 

meee 1/2 + 1/3 + 2. + Y(X, — 1). | 
Resulting in Equation 2.10. 


I 


\@ 


1 


eg 2 ee 1/(X = 1) for X, 2 2 (eqn 2.10) 
0 for x, 


~~ 


Substituting Ge an unbiased estimate of A, into the reliability equation yields an 
estimate of single phase reliability, Equation 2.11. Equation 2.11 can be used to 
estimate system reliability when testing is performed until first failure. 

As has been shown, Y; is an unbiased estimate of A; but, this does not ensure 


Equation 2.11 will provide an unbiased estimate of system reliability. To resolve this 
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R=1- ej (eqnuZ. un) 


question Monte-Carlo simulation was used to evaluate the properties of the 
Exponential Single Phase Estimate and compare it with the MLE. 

The algorithm described below was used to evaluate the Exponential Single 
Phase Estimate, Equation 2.11. First a uniform (0,1) random number was generated. A 
table look-up algorithm was used to convert the uniform (0,1) random variable to a 
geometric random variable with reliability R. Estimates of system reliability, based on 
the geometric random variable, were computed using both the Exponential Single 
Phase Estimate and the Maximum Likelihood Estimate. The procedure was then 
replicated a large number of times. The running average of the estimates for each 
estimator was updated after each replication. The results of the simulation are shown 


in Table 6 for ten-thousand replications. 


eal Blom 6 


ESTIMA Te-exX Tr EGE VALUE 
FOR MLE & EXPONENTIAL ESMIM ATs 


THEORETICAL RESULTS SIMULATION RESULTS 
R _ E[MLE] MEAN MLE MEAN EXP. EST. 
I 052 053 066 
2 107 107 nee 
E 168 170 207 
4 234 234 281 
5 50g 305 B6i 
& 389 388 452 
i A484 A84 552 
g 598 598 664 
9 744 744 798 
99 O55 954 968 
999 993 994 996 
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The simulation generated mean for the MLE is very close to its theoretically 
derived value verifying the accuracy of the simulation. Comparing the simulation 
generated mean estimate for the Exponential Single Phase Estimator to the actual 
reliability leads to the conclusion that the estimate is conservatively biased, but less 
biased then the MLE. As is the case with the MLE, the amount of bias in the 
Exponential Single Phase Estimate is a function of the actual reliability. 

Constant system reliability is necessary for the Exponential Single Phase 
Estimate to accurately predict reliability. Because of this property, the estimator must 
be modified to accurately predict the changing system reliability inherent in a reliability 
growth pattern. As was the case with the MLE, the modification should make it 
possible to use all available test data when computing a reliability estimate. The 
remaining reliability growth models explore alternative modifications to the. 
Exponential Single Phase Estimate. The first modification employs linear regression to 
estimate the parameter A. The second modification involves taking the weighted 
average of the current single phase estimate and the preceding phase reliability estimate 
to produce an estimate of current phase reliability. 

3, Exponential Regression Discrete Reliability Growth Model 

The Exponential Regression Reliability Growth Model was developed by W. 
'M. Woods from the Exponential Single Phase Reliability Model [Ref. 4]. The model 
studied here is identical to the original model with the exception of applving failure 
discounting. The model can only be applied in situations where testing is performed to 
a fixed number of failures and must have attribute data. Any failure discounting 
method may be used with the model; but, the discounted failure data must be returned 
to its original form, an integer number of failures and successes. To apply the model, 
One must record the data necessary to apply the chosen failure discounting method, the 
total number of failures and successes, and the number of successful trials between 


each failure. The equation 


R=1l1-—e4 
where 
A=a+ Bk 


and k denotes the testing phase is used to compute system reliability 
Rr = | ace 6 k) 


mee 1, 2, 3, ... 
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Estimating the expression a + Bk at each phase allows the model to track changing 
reliability. The model is further modified to include the possibility of more than one 
failure in a given phase. Let F, equal the total number of failures in the k th phase and 
j] equals the failure number in phase k such that] = 1, 2, ..., Fy. Nix equals the 
number of trials between the (j — 1) St and the j th failure, including the } th failure, in 
phase k. As in the Exponential Single Phase Estimate, Equation 2.10, still represents 
an unbiased estimate of ( a + B k) for each set of trials Nik: To avoid confusion with 


the additional subscript, Equation 2.10 becomes Equation 2.12 


Yok =P 12 eee alee =a for x, aoe (eqn 2.12) 
= 0 for Xx, 


I 
~— 


Estimates of a and 8 for each phase are developed using the techniques of linear 


K 
ei one 
j=l 
80 = _ (eqn 2.13) 
k 
yi Ke 
j=l 


regression yielding Equations 2.13 and 2.16. Equation 2.13 requires the computation 
of Ys , Equation 2.14 , and K. , Equation 2.15. 


Di, = OGee 9 Yoy eee oe rae {eqn 2.14) 
Ke (02 eee) (eqn 2.15) 
g, = Y.. — B, K. (eqn 2.16) 


30 


fee ity t Y5 +... +t Vy YK (eam 2-17) 


Substituting the estimates for a, and B. into the reliability equation yields the estimate 


of reliability for phase k, Equation 2.18. 


tI 


B= 1 — e(% + Bb) k>1 (eqn 2.18) 
=1-e% el 


A separate definition of the case where k= 1 is necessarv because regression techniques 
require a minimum of two data points. [Ref. 3: pp.1-4] 

The only modification made to the original model is the addition of failure 
discounting. After each phase, the chosen failure discounting method is applied to the 
test data. The fractional failures that result must be adjusted to unity bv adjusting the 
number of trials, Equation 2.7. Computation of Vix requires the total number of trials 
Bemretuimed to an integer value. Therefore, the adjusted trials are rounded to the 
nearest integer. Estimates for B, Equation 2.13, and a, Equation 2.16, are then 
computed. These values are substituted into Equation 2.18 to obtain the estimate of 
system reliability for phase k. 

Because the parameters a and § are computed using linear regression, the 
model has the ability to track changing reliability even if no failure discounting is 
applied. Even so, it may take numerous phases and a great deal of data for the 
Exponential Regression Model to accurately predict reliability. Adding failure 
discounting to the model should decrease the amount of data required for an accurate 
estimate of svstem reliability. 

The Exponential Regression Model can predict a negative system reliability. 
This occurs most often in the initial testing phases and at low reliabilities. Should this 
occur, the estimate should be set to zero. 

An example will help clanfy the application of the model. Using the 
discounted test data trom Table 2, the estimate of phase seven reliability is computed 
melable 7, 

Notesthe Yix column was deleted because each phase consists of a single 
failure and Yix = Y.,. This example also demonstrates the results of applying 
improper discounting parameters for the situation. Failure discounting caused the 
estimator to further overestimate the actual reliability. Had no discounting been 
applied, the regression estimate would have yielded R. = .836. 


by 


TABLE 7 
EXPONENTIAL REGRESSION ESTIMATE COMPUTATIONS 


k ADJ. TRIALS Gd (Keane (key 
2 1.0 — 3.0 9 
2 2 1.0 20 4 
3 2 1.0 ale) 
4 3 [5 0.0 0 
5 5 2.083 2.083 
6 5 2.083 4.166 4 
7 8 DSI 7.779 9 
» 11.259 8.023 28 


K. = 4.0 

Yee= 11.259/7 = aes 

B. = (8.028 + 28) = .287 

G@. = 1.608 — (.287 X 4) = .460 

(a. + B. K) = .460 + (.287 X 7) = 2.469 
Rey | eek 0) ne 
R, = .78 


The last remaining reliability growth model is also derived from the 
Exponential Single Phase Estimate. A weighted average technique is used to allow the 
estimator to track changing reliability. This estimator is described in the next section. 

4. Weighted Average Discrete Reliability Growth Model 

The Weighted Average Model tracks changing reliability by taking a weighted . 
average of the two most recent reliability estimates. This model is only applicable to 
situations when testing is performed until a fixed number of failures have been 
observed and attribute data is available. A slightly modified version of the Exponential 
Single Phase Reliability Estimate is used in the model. The current phase reliability is 


estimated using a weighted average of the previous phase reliability estimate and the 





current single phase estimate. The previous phase estimate and the current single phase 
estimate are weighted by the fraction of total trials to date used in the computation of 
each estimate. As with the Exponential Regression Estimate, this model will track 
changing reliability without the use of failure discounting; but, like most weighted 
averages, the model may lag actual growth by one or more phases. Failure discounting 
is used to correct the lag and reduce the number of trials necessary to compute an 
accurate estimate. 

Equation 2.11 is applicable when testing to first failure. A slightly different 
version of the estimator is applicable to situations where more than one failure has 
been recorded. To apply this estimator, a running total of the number of successful 
trials to date, Sy , and failures to date, Sig is maintained for each phase k. These totals 
are used in Equation 2.19 to compute the current single phase reliability estimate, 
Bop: 


i= Ee: 


Esp. & feqmacsla) 
ee I/F, + ee eee (ts 5) — | | 


For this model, running totals of failure and success data are recorded for 
each observed failure cause. As each trial is performed, the number of successes for 
each failure cause, except the current failure cause, is incremented by one. Should a 
previously recorded failure cause reoccur, its failure total is incremented by one. The 
system values for F, and S, are then selected from the individual failure cause having 
the largest number of failures to date. 

Once the single phase estimate 1s computed, the estimate of svstem reliability 


is computed by taking the weighted average of the previous phase reliability estimate 


N N 
= 1 cay. 
R. = \ ora + yy Esp, — Jo (eqn 2.20) 
a Esp. k= 1 


and the current single phase estimate, Equation 2.20. 


Where N equals the total number of trials recorded to date. N, equals the total 
number of trials recorded in previous phases. N, equals the number of trials recorded 
in phase k. Note the sum of N, and N, equals N. 
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Failure discounting is applied to the model in much the same way as it was in 
the Exponential Regression Model. After a discounting method has been applied to 
the data, fractional failures are returned to unity by adjusting the number of trials, 
Equation 2.7. The adjusted number of trials is then rounded to the nearest integer. It is 
important to note, when failure discounting is applied to this model S, must be 
recomputed from phase one at each phase boundary because the recorded number of 
successful trials for each previous phase may be altered by the discounting routine. 

An example of the model’s application should help clarify the required 
computations. Using the discounted failure data from the example of the Straight 
Percent Discounting Method leading up to Table 2, at the end of phase one the sum of 


failures and successes for the failure cause Its listed below. 


PATI INES SUCEESSES 
CAC ree l | 


Solving for the single phase estimate and the reliability estimate yield 


Cc, = 1.0 


Il 
_ = ale 
a Esp, = | Gi == 057 ie. 


After the second phase data is included and the failure discounting method applied, the 


sum of failures and successes for each failure cause js listed below. 


EAL Kes SUCCESSES 
CAUSE A | 2 
CAUSE 3B J Z 


From the failure data, F, equals one and S, equals two yielding €, equal to 1.5. 


Applying the remaining model yields the second phase reliability estimate &.,. 


Bop, = 1 - e(7/2) = 7769 
Ni =2 
N= 1 
N = 3 


R, = (2/3) B, + (1/3) By 
(2/3).6321) + (1/3)(.7769) 
6804 


Il 
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The computations are repeated for each phase of the testing program. As 
mentioned previously, when failure discounting alters the number of successful trials, 
S 


recomputed to solve for the current phase reliability estimate. As this simple example 


,, In a previous phase, the reliability estimates for all previous phases must be 


clearly shows, this reliability growth model is computationally intense. Programming 
the model on a computer may be necessary to compute reliability estimates for large 
data sets. 

Two failure discounting methods and three discrete reliability growth models 
have been described tn this chapter. Evaluation of each model is verv difficult because 
the expected value and variance of the estimates, when discounting is applied, become 
mathematically untractable. It is also not possible to evaluate the models using actual 
test data because the true reliability at each phase is unknown. The following chapter 
discusses a Monte-Carlo simulation written to evaluate the performance of each of the 


faulure discounting methods and discrete reliability growth models. 
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Ii. DISCRETE RELIABILITY GROWTH SIMULATION 


Evaluating reliability growth estimators including reliability growth models, is 
often difficult. True svstem reliability 1s unknown, rendering actual test data useless for 
evaluating an estimators performance. When the mathematics is tractable, the 
properties of the estimator can be derived and used to evaluate the estimators 
performance. However, many estimators are too complex to allow mathematical 
solutions for the closed form of their properties. For this group of estimators, 
computer simulation is one of the only evaluation techniques available. The addition of 
failure discounting to an estimator makes mathematical solutions very difficult if not 
impossible. The reliability growth models introduced in this study fall in the category 
requiring simulation as an evaluation tool. 

A Monte-Carlo simulation was written to evaluate the properties of the three 
reliability growth models introduced in Chapter II. The simulation builds a reliability 
growth pattern and generates test data from the known system reliabilies using the 
geometric distribution. Each of the reliability growth models are then applied to the 
simulation generated test data. Repeating the process many times for a particular set of | 
known reliabilities will provide a good estimate of both the mean and variance of each 
estimator as well as provide information about the underlving distribution of the 
estimate. These statistics can then be used to evaluate each reliability growth model's 
performance. 

Several modeling problems had to be solved before the simulation could be built. 
A method for generating a realistic reliability growth pattern had to be developed. This 
method must closely model the process of testing equipment, implementing design 
improvements or “fixes” and continued testing. It also must providing the flexibility to 
test numerous reliability growth patterns. Once the actual system reliability in each 
phase is established from the growth pattern, test data of the form the number of trials 
up to and including failure must be generated. Because such data are geometrically 
distributed when R is fixed, this process is relatively simple. A relistic method must also 
be developed for determining the cause of each failure. This step proved to be the 
most difficult. 
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As with all simulations, several modeling assumptions were necessary. The 
following section discusses the assumptions and their impact on the simulation. The 
remainder of the chapter describes the development of the simulation algorithm and its 


capabilities and limitations. 


A. MODELING ASSUMPTIONS 
Building an algorithm that will generate realistic test data requires several 
simplifying assumptions about the actual test process. These assumptions can be 
broken into three categories. They are: 
1. General assumptions 
2. Assumptions concerning the true reliability growth pattern 
3. Assumptions concerning failure cause determination 
The last two categories correspond to the two steps in generating test data, generating 
the actual reliability growth pattern and computing test data from the growth pattern. 
Each assumption 1s listed and discussed by category. 
1. General Assumptions 
The general assumptions narrow the scope of testing considered to those 
scenarios which apply to the reliability growth models evaluated. The two required 
assumptions are: 
1. All testing is performed to a fixed number of failures. 
2. All test results are recorded as attribute data. 
These assumptions ensure the test data produced by the simulation will be compatible 
with the discrete reliability growth models developed in the study. The models can only 
be applied to discrete or attribute data where testing is performed until a fixed number 
of failures are observed. 
a Reliability Growth Pattern Assumptions 
Defining the mechanisms which cause an items’ reliability to change from 
phase to phase is necessary before an algorithm to generate a realistic reliability 
growth pattern can be written. Several assumptions were necessary to properly define 
these mechanisms. They are: 
I. The reliability growth pattern is non-decreasing. 
2. System reliability changes only at phase boundaries. 


3. Equipment improvements are implemented immediately after a phase ends and 
' before any further testing. 


By 


4. Only design weaknesses causing failure during a phase are corrected at the end 
of a phase. | 


5. Each design improvement removes a fixed fraction of the probability of 
reoccurrence for the corresponding failure cause. 


The first assumption limits the shape of the generated growth patterns to non- 
decreasing curves. Because one of the objectives of development testing is to improve 
system reliability, this assumption is logical. However, actual reliability growth may 
have periods of decline when “fixes” to design problems actually decrease reliability. 
Over the entire testing period, even these curves are generally increasing. The worst 
case this assumption allows is no growth over the testing phases. This should be 
adequate for determining adverse growth characteristics of the reliability growth 
models. 

Assuming system reliability remains constant throughout a testing phase 
makes the model consistent with the previous definition of a testing phase. This 
assumption means no design changes or any other improvements are made to an item 
on test within a phase. Because a phase can be defined as any number of failures 
greater than or equal to one, this assumption should have little impact upon the 
accuracy of the simulation. 

Assuming all fixes are implemented after a test phase ends and before the next 
phase begins is closely related to the previous assumption. This assumption does 
preclude accurate modeling of long term fixes whose implementation are postponed 
while testing continues. The impact of this limitation should be minimal. 

Assumption four asserts that the only known weaknesses in system design are 
those which have caused system failure. This is not always the case. Suspected 
weaknesses may be corrected along with those weaknesses confirmed by system failure. 
The simulation is unable to model this situation. Even though this deviates somewhat 
from reality, most major improvements to system reliability should come from failure 
identified weaknesses. The intent of the simulation is only to generate a realistic 
-rehability growth pattern not to duplicate an actual growth pattern. Therefore the 
simulation results should be adequate to accomplish the task for which the model was 
designed. 

The last assumption in this category deals with the mechanism used by the 
simulation to model improvements in system design. It asserts that all fixes remove a 
constant fraction of the probability of occurrence for the failure cause of interest. In 


reality many factors determine how effective a particular fix will be and all fixes are not 


equally effective. This method also decreases the probability of occurrence more for a 
failure cause with a high probability of occurrence than for a failure cause with a low 
probability of occurrence. This also is unrealistic. Despite the shortcomings of this 
simplification, it does provide a simple way to model reliability improvement due to 
design change and will produce a growth pattern that is adequate for evaluating the 
reliability growth models studied. 

The final category of assumptions deal with generating test data from a known 
system reliability and determining the cause of each failure. They are discussed in the 
mext section. 

3. Failure Cause Determination Assumptions 

Realistically modeling failure causes is critical to the evaluation of the failure 
discounting methods. The technique used to model failure cause determination must 
also be incorporated into the technique used to model reliability growth because the 
two processes are closely related. The assumptions made in modeling failure cause 
determination are: 

1. There exists a finite number of possible failure causes. 

2. Each failure cause has a fixed probability of occurrence in each phase. 

3. System reliability can be modeled as a series system of the failure causes. 
4. Each failure cause is stochastically independent of the other failure causes. 

Assuming a fixed number of failure causes may be an oversimplification of 
reality but should not invalidate the model. Even though the possible number of failure 
causes is most likely infinite, only a finite number of these causes with relatively high 
probabilities of occurrence are of interest. The number of possible failure causes 
considered by the simulation must be specified as an input to the model; but, the 
number of causes input is only limited by the capacity of the computer on which the 
model is run. Therefore, the simulation is flexible enough to model nearly any scenario 
of interest. | 

Fixing the probability of occurrence for all failures causes within a single 
phase is a direct result of fixing the system reliabilitv for all trials in a single phase. 
This assumption will not further restrict the ability of the algorithm to model reality. 

An assumption 1s needed to interrelate failure cause determination and system 
reliability growth through design improvements. Assuming the component being tested 
may be modeled as a series system of failure causes meets this requirement. Svstem 


reliability at any phase is a function of the probability of occurrence for the specified 
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failure causes. As such, implementing a design improvement reduces the probability of 
occurrence for a failure cause which in turn increases the systems reliability. This 
assumption also makes it possible to stochastically determine the cause of failure. The 
mathematical derivation of this technique will be presented in a later section. 
Modeling the failure cause as a series system does not depart far from reality because 
the occurrence of a failure cause implies system failure. Once the assumption of a 
finite set of failure causes is accepted, modeling them as a series system does not 
further restrict the models representation of reality. 

The final assumption requires that each failure cause be stochastically 
independent. This means that the causes of failure are not related. The occurrence of 
one cause does does not influence the probability of occurrence for any other cause. If 
the failure causes are properly defined, this assumption should not limit the accuracy of 
the model. 

The following section discusses the simulation algorithm built from these 


assumptions. 


B. SIMULATION ALGORITHM DEVELOPMENT 
: The algorithm developed for this simulation can be broken into the four sections. 
They are: 

1. Generate a reliability growth pattern. 


2. Generate test data, including the cause of failure, from the reliability growth 
pattern. 


3. Apply the failure discounting methods and reliability growth models to the 
simulation generated data. 


4. Compute statistics used to evaluate the performance of each model. 

The first two steps of the algorithm along with the output: statistics will be 
discussed in detail in this section. The methodology used to apply each of the 
discounting methods and reliability growth models was discussed in Chapter II and will 
receive little attention here. With few exceptions, the actual content of the Fortran 
code used in the simulation will not be discussed. A documented copy of the code is 
included at Appendix A for the interested reader. 

1. Simulation of a Reliability Growth Pattern 

The reliability growth pattern built by the simulation models the actual 
process of testing and improving an item in a development program. Several user 
inputs are required at this stage of the simulation. They are: 


1. number of testing phases 
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number of failures in each testing phase 
number of possible failure causes 


the first phase probability of occurrence for each possible failure cause 


i) oa Se 


growth fraction (F.) 

Wince these mamcrere are in place, the simulation computes the first phase 
system reliability from the probability of occurrence for each possible failure cause. The 
number of trials up to and including failure and the failure cause are then computed for 
all failures in the first phase. The algorithm used.to generate svstem reliability and the 
failure data will be covered in the next section. 

After the failure data for the first phase has been generated, the probability of 
occurrence for each failure cause which occurred in the first phase is recomputed using 
the growth fraction. Assume the first failure occurred due to cause A, the new 


probability of occurrence, Pr (occurrence), will be computed using Equation 3.1. 
Pr (occurrence), = Pr(occurrence) xX (1 — Fy) feqnean) 


This equation is applied at the end of each phase to those failure causes that occurred 
in the current phase. If a failure cause is responsible for multiple failures in the phase, 
Equation 3.1 is applied only one time. This procedure simulates identifying a system 
weakness through testing and implementing a design improvement. The procedure is 
repeated at the end of each testing phase. 

The result of the algorithm is a stochastically determined reliability growth 
pattern that simulates actual developmental testing. A test is performed, weaknesses in 
the system are identified, corrective design changes are implemented and testing 1s 
‘continued. This algorithm also has the advantage of interrelating the probabilities of 
occurrence for each failure cause and the system reliability. This will be useful in 
determining the failure cause. By changing the growth fraction, the rate of reliability 
growth can be varied from no growth to rapid growth. 

2. Generating Test Data From Known Reliability 

Two problems must be solved before an algorithm that generates test data can 
be developed. The system reliability must be determined from the probability of 
occurrence for each failure cause. A technique for generating the number of trials up to 
and including failure from Known reliability that identifies the failure cause also must 


be developed. The solution to both problems lies with the assumption that system 
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reliability can be modeled as a series system of the failure causes. In a discrete series 
system the number of trials up to and including system failure will be the minimum of 
the number of trials up to and including failure for each sub-component. Let M equals 
the number of trials up to and including failure for the system and X, where 
i=1,2,..,j denotes the number of trials up to and including failure for them th 
subcomponent, here a failure cause. If the } components form a series system, M may 
be expressed as 
M = MIN (Ny X45) 

Because the probability of occurrence is fixed throughout a phase for each failure 
cause, X, is distributed as a geometric random variable with parameter R,. From an 
assumption, the X,’s are also mutually independent. It can be shown for a geometric 
random variable, X,, that 

ie (OG ial) lke 
The distribution for M 1s derived as follows. 

M = MIN [ Xp Xa ] 
Pr(M > nl = Prix = ae ae Xj >n| 
= Pr [Xue 1) ielee ea) oe Pr [X; > n] 
= Reva R," Xa R* : 
ol Roi OS I 
Therefore M is also distributed as a geometric random variable with the parameter 
R=( RK) *~ hy ee R; ). 

The system reliability at each phase is computed as the product of the 
compliments of the probabilities of occurrence for each possible failure cause. Because 
of this computation the actual simulation input is the compliment of the first phase 
probability of occurrence for each failure cause. As the probability of occurrence for 
each failure cause is recomputed from phase to phase, the system reliability is also 
adjusted. 

The number of trials up to and including failure for a geometrically distributed 
random variable with reliability R can be generated from a uniform (0,1) random 


variable, U, using Equation 3.2. (Ref. 5]. 


X = INT[1 + (In(U)/In(R) )] (eqn 3.2) 
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Equation 3.2 combined with the definition of a series system provide the solution to 
generating the number of trials up to and including failure while accounting for the 
cause of failure. 
For each failure generated by the simulation, Equation 3.2 1s used to generate 
a geometric random variable for each failure cause using the compliment: of the 
probability of occurrence as the parameter. The minimum geometric random variable 
over the failure causes becomes the number of trials up to and including failure for the 
system. The cause of system failure is recorded as the failure cause producing the 
minimum geometric random variable. Generating the test data in this fashion allows 
the reliability growth pattern and test data to be stochastically generated yet 
interrelated much as they are in nature. A summary of the steps required to compute 
the reliability growth pattern and the test data follows. 
Input the required data 
2. Generate phase failure data including the number of trials and the failure cause. 


Recompute the probability of occurrence for the failure causes that occur in the 
phase. 


4. Return to step 2 until all phases are complete. 

Once the test data has been computed from the known reliability in each 
phase, the failure discounting methods and reliability growth models can be applied to 
the test data. The resulting reliability estimates can then be evaluated against the 
known system reliability. 

a3. Application of the Discounting Methods and Reliability Growth Models 

The discounting methods and reliability growth models are applied to the 
generated test data in much the same way as they were described in Chapter II. Each 
of the three reliability growth models are applied to each set of test results. However, 
the. simulation user must specify one of the two failure discounting methods because 
the simulation can not apply both simultaneously. The maximum likelihood estimate 
without discounting is also computed for each phase, independent of all other test data. 
This allows a comparison between the standard reliability estimator and the reliability 
growth models. The simulation produces an estimate of system reliability from each 
reliability growth model for each testing phase. The single phase MLE and actual 
system reliability are provided as a basis for comparison. 

_ One minor departure from the application of the reliability growth models as 
described in Chapter II was necessary. For large numbers of trials up to and including 


failure, N, computations of the form 
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Y= 1+ (1/2) +... + fI/(N-1)] 
required extensive computer processing time to solve recursively. To improve program 
efficiency, the values for Y up to and including one-thousand were computed exactly 
and stored in an array. For values of N greater than one-thousand non-linear 
regression was used to estimate the value of Y. Equation 3.3 provided an adequate 


estimate with an R* value of .99635. 


Y = 3.55445 x N(I1/100) (eqn 3.3) 


COMPARISON OF ACTUAL VS ESTIMATED VALUES OF Y 


ESTIMATED 


ACTUAL 


] ! 
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Figure 3.1 Regression Estimate of Y. 


Figure 3.1 compares the actual Y value with the regression predicted value for large N. 
Both the Weighted Average Model and the Exponential Regression Model are affected 
by the approximation. The approximation has minimal impact upon the computation 


of model estimates. Values for N above one-thousand are rare even at high reliability 
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and values above 15,000, the accurate extension of the approximation, are verv rare. 
Should a value of N larger than 15,000 occur, some error will be introduced into the 
model estimate, and this error will increase with N. Because these values are rare, this 
error is not considered significant. 


Further research, after the simulation was written, has yielded a more accurate 


Y= eeros.N) oN) =e NEN 1) — 1/(V2NON + LGN + 2)] (eqn 3.4) 
C = Euler’s Number = 0.5772156648 


approximation for Y. This series solution is shown in Equation 3.4 [Ref. 6]. This 
equation should be incorporated into the simulation to improve accuracy in future 
runs. 

The estimates resulting from the application of the reliability growth models 
are retained through the desired number of replications to allow the computation of 
descriptive statistics. These statistics aid in evaluating each growth model. 

4. Statistical Summary of Mode! Results 

Several descriptive statistics are generated by the model that aid in evaluating 
the performance of each reliability growth model. They include the mean and standard 
deviation of the reliability estimates at each phase for each of the growth models. A 
confidence interval is also computed for the mean estimate at each phase. | 

Replications of the simulation are required to compute the descriptive 
Statistics. After the first replication, the actual system reliability in each phase is fixed. 
At each replication, new test data is generated from the fixed reliability growth pattern 
and each of the reliability growth models is applied to the data. The estimates 
generated by the growth models are retained at each replication. When the desired 
number of replications have been completed, the sample mean and sample standard 
deviation for each estimator are computed using standard formulas. A 95% confidence 
interval for the mean estimate is computed for each phase reliability estimate with the 
assumption that each estimate is normaily distributed. The assumption of normality is 
not valid in every case but the confidence interval is included as a tool for comparison. 
Further analysis of the reliability growth models can be accomplished by saving each 
estimate in an output file. The models user can then analyze the data as desired. 

Changing the simulations’ input parameters makes it possible to analyse a 
wide variety of testing situations. These capabilities are discussed in the following 


section. 
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C. CAPABILITIES OF THE MODEL 

Within the limits of the assumptions stated earlier, the simulation is very flexible. 
Any number of failure causes may be input into the model. By varying the phase one 
probability of success for each failure cause, the growth fraction and the number of 
phases a wide range of reliability growth curves may be explored. The random number 
generator seed is also user specified allowing the exact string of random numbers and 
test data to be duplicated. In this way, various discounting methods and parameters 
may be evaluated using the exact same data. 

The test-fix-test scenario may be modeled by specifying a single failure per phase. 
The test-find-test methodology may be simulated by specifying multiple failures in each 
phase. There is no requirement that each phase have an equal number of failures. 
Therefore a mixed scenario, test-fix-test and test-find-test, may be simulated. 

Each of the discounting methods described in Chapter II may be applied to the 
data using the full range of input parameters. Proper parameter selection for the 
discounting method will cause no discounting to occur. In this manner each of the 
reliability growth models may be evaluated without failure discounting applied. 

The simulation provides a tool for evaluating the reliability growth models 
against a known system reliability growth pattern. When large numbers of replications 
are run, information concerning the expected value and variance of each estimator may 
be obtained. The combinations of failure discounting parameters, reliability growth 
models and anticipated growth patterns may also be evaluated. The following chapter 
discusses the results of numerous runs of the simulation with varying growth patterns 


and failure discounting parameters. 
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IV. EVALUATION OF RELIABILITY GROWTH MODELS 


The discounting methods and reliability growth models were evaluated using the 
Monte-Carlo simulation described in Chapter III. The results from numerous runs of 
the simulation with varving reliability growth patterns, failure discounting methods and 
failure discounting parameters were reduced to graphical output for evaluation. 

The purpose of the evaluation was to determine general characteristics of each 
reliability growth model. The number of simulauon runs was not adequate to prove 
that conclusions reached here will hold for all cases. Also, no fast rules for model 
selection or failure discounting parameter selection were developed. Stull, many 
characteristics of the different reliability growth models were apparent from the 
simulation runs completed. In particular the answers to the following questions were 
sought. 


1. How well does the reliability growth model track the actual reliability growth 
pattern? | 


2. How did the choice of failure discounting parameters affect the estimates of 
reliability? 


3. How variable were the estimates provided by each of the models? 

It was impossible to evaluate all possible reliability growth patterns that can be 
produced by the simulation. Five widely different reliability growth patterns were 
chosen to be representative of the possible range. Constant reliability cases were 
included to identify models that predict reliability growth, were in fact, none exists. The 
five reliability growth patterns chosen are described below. 

1. low reliability with no growth 

2. high reliability with no growth 

3. low initial reliability with moderate growth 

4. moderate initial reliability with moderate growth 
5. low intual reliability with rapid growth 

All simulation input parameters were kept fixed to insure the different simulation 
runs could be compared. The number of phases was held constant at ten with one 
failure per phase. Five failure causes were used and each simulation was replicated one- 
hundred times. The seed for the random number generator was kept constant for all 
sumulation runs insuring the same string of random numbers and, therefore, identical 


test data was generated for each growth pattern. 
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Each reliability growth model is evaluated separately in the following sections. 


The graphical results of the remaining simulation runs are included in Appendix B. 


A. GENERAL RESULTS 

Characteristics common to all the relability growth models are stated here for 
convenience. 

The variability of each of the estimators decreased with increasing phases. This is 
due to the increased data available to the model with each additional phase. 

The accuracy of each estimate increased and the variability of each estimate 
decreased with increasing reliability. This is a function of testing until a fixed number 
of failures have been observed. As reliability increases, the average number of trials 
required to produce failure will also increase. Therefore, as reliability increases, more 
data 1s available for the model improving the accuracy of the estimate and decreasing 


the estimate’s variability. 


B. WEIGHTED AVERAGE RELIABILITY GROWTH MODEL 

The Weighted Average Reliability Growth Model proved to be ineffective as an 
estimator of changing reliability. The model consistently overestimated the actual — 
relability even when no failure discounting was applied. Figure 4.1 plots the mean 
estimate from each model at each phase. Clearly the mean phase estimates for the 
Weighted Average Model overestimated the actual reliability at each phase. The 
addition of failure discounting only amplified the tendency to overestimate as seen in 
Figure 4.2. Even though the variance of the estimates decreased rapidly with 
increasing phases, the model is not useful in its current form. Therefore, the Weighted 


Average Model was not considered in any further analysis. 


C. MAXIMUM LIKELIHOOD ESTIMATE WITH FAILURE DISCOUNTING 

The MLE With Discounting Model accurately tracked a wide range of reliability 
growth patterns. With proper selection of the failure discounting method and 
parameter, this model tracked each of the reliability growth patterns tested. Figures 
4.2, 4.3, 4.4, 4.5, and 4.6 display the model!’s performance for each of the five reliability 
growth patterns tested. The model is very sensitive to the selection of the failure 
discounting method and parameter. In general, the model underestimates actual 
reliability in the early phases and slowly converges to the actual value with increased 
test data. 
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Figure 4.1 Reliability Growth Models © 
Low Constant Reliability 
No Discounting Applied. 


The choice of failure discounting method and parameter is critical to the MLE 
With Discounting Model. Because the Lloyd Discounting Method does not have a 
discounting interval, it generally reduces the discounted failure value more rapidly than 
the Straight Percent Discounting Method. This results in rapid growth of the reliability 
estimates. The Standard Discounting Method with the discount interval set at one and 
the proper choice of the discount fraction will provide nearly identical results to the 
Lloyd method. For all the reliability growth patterns tested, except low initial reliability 
with rapid growth, the Lloyd discounting Method caused the model to overestimate the 
actual reliability. The overestimation was particularly pronounced for low constant 
reliability where the Llovd Discounting Method forced the model to overestimate 
reliability even with y as large as .99999, Figure 4.7. Because of this characteristic, the 
Lloyd discounting Method is not recommended for use with the Maximum Likelihood 


Estimate. 
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Figure 4.2 Reliability Growth Models 
Low Constant Rehability 
Std. Discounting Applied. 


With the addition of the second parameter, the discount interval, the Straight 
Percent Discount Method is more flexible than the Lloyd Method. Even so, it has the 
potential to overestimate the system reliability. The choice of model parameters is 
critical to the model’s performance. Figure 4.8 demonstrates the change in model 
estimates with varving discounting parameters. Several factors influence the proper 
choice of parameters. Three of the most important are the amount of testing to be 
performed, estimates of system reliability before testing begins and how rapidly 
reliability is expected to grow throughout the development and testing program. No 
rules have been developed to help select the proper discounting parameters. Experience 
with the model, good engineering judgement and possibly simulation are necessary to 
select a combination of discounting parameters that will cause the model to accurately 
track the actual reliability growth pattern. Even an apparently good choice of failure 


discounting parameters may result in erroneous reliability estimates if the true 
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Figure 4.3. Reliability Growth Models 
High Constant Reliability 
Std. Discounting Applied. 


reliability does not grow as expected. Therefore, when using this model, frequent 
engineering evaluations may be necessary to insure design evolution is progressing as 
expected: 

| Reliability estimates from the Maximum Likelihood Estimate With Discounting 
Model exhibited the smallest variance of the models evaluated. Figures 4.9 and 4.10 
are box plots of the one hundred reliability estimates at each testing phase for two of 
the reliability growth patterns. Comparison with other models may be made from the 
additional box plots included at Appendix B. The reliability estimates from all three 
models exhibited large variance in the first few phases of testing. The variance of the 
estimates gradually decreased as more test data became available. With few exceptions, 
the MLE With Discounting exhibited the smallest variability over the entire range of 
phases for each reliability growth pattern tested. This reduced vanance was 


particularly pronounced in the early phases and at low reliability. 
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Figure 4.4 Reliability Growth Models 
Low Reliability Moderate Growth 
Std. Discounting Applied. 


The MLE With Discounting Model can accurately track most reliability growth 
patterns when the Straight Percent Discounting Method is used. It does so with equal 
or less variance than the other models evaluated. However, the model is very sensitive 
to the choice of of failure discounting parameters and may either over or under 


estimate the actual reliability if the parameters are not chosen correctly. 


D. EXPONENTIAL REGRESSION MODEL 

The Exponential Regression Model has the ability to accurately track most 
reliability growth patterns, Because it uses a combination of linear regression 
techniques and failure discounting to track changing reliability, it reacts quickly to 
changes in actual reliability. Figures 4.2, 4.3, 4.4, 4.5, and 4.6 demonstrate the model's 
performance with each growth pattern. It also is much less sensitive to the selection of 


failure discounting parameters than is the MLE With Discounting Model. However, 
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Figure 4.5 Relhability Growth Models 
High Reliability Moderate Growth 
Std. Discounting Applied. 


the variance of the early phase estimates is much greater than the estimates from the 
MLE With Discounting Model. 

This model requires several phases, usually at least four, to accurateiv estimate 
system relabilitv. The number of phases required is a function of the shape of the 
reliability erowth pattern. More phases are required for an accurate estimate at low 
system rehability than at high reliability. The model also appears to track changing 
reliability more accurately than constant reliability. 

The Exponential Regression Model is relatively robust with respect to the 
selection of a failure discounting method and parameters. Still, as was the case with the 
MLE With Discounting Model, the Llovd Discounting Method tends to cause the 
model to overestimate actual reliability as is demonstrated by Figure 4.11. Figure 4.12 
demonstrates how robust the model can be to changing parameters when the Straight 


Percent Discount Method is applied. Because the Straight Percent Discounting 
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Figure 4.6 Reliability Growth Models 
Low Rehabilitvy Rapid Growth 
Std. Discounting Applied. 


Method can produce nearly equivalent results and is much more flexible, use of the 
Llovd Discounting Method is not recommended with this model. 

Again, no fixed rules exist for chosing the failure discounting parameters of the 
Straight Percent Discounting Method. Experience with the model, engineering 
judgement and possibly computer simulation will be necessary to select the proper 
combination of the discount interval and discount fraction. Even though the model 1s 
fairly robust concerning this selection. it is possible to choose parameters that will 
force the model to overestimate reliability. Tne danger of this occurring is greatest 
when expected growth in the system reliability does not occur. | 

The reliability estimates produced by the exponential regression model exhibit a 
higher variance than those from the MLE With Discounting Model. This increase in 
variance is most pronounced in the early testing phases. Figures 4.13 and 4.14 are box 
plots of the one-hundred reliability estimates at each phase. They demonstrate the 


variability of the model. 
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Figure 4.7 MLE With Discounting 
Lioyd Discounting Method. 


As with any regression, several points are required to stabilize the regression 
parameter estimates. This causes highly variable responses in the first two or three 
phases of testing. This variance is largest when reliability in the early phases is low. In 
later phases and at higher reliabilitv, the variance of estimates from: the Exponential 
Regression Model compares favorably to those of the MLE With Discounting Model. 

The Exponential Regression Mdodel closely tracks most reliability growth 
patterns. [t is best when more than three test phases are planned. The most accurate 
results were obtained when the model was used with the Straight Percent Discounting 
Method. The modei is relatively robust with respect to the selection of discounting 
parameters; but, may still over or under estimate actual reliability if a bad combination 
of parameters is selected. The variance of reliability estimates in the early testing 
phases is large and makes the model undesirable for use with less than four testing 


phases. 
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Figure 4.8 MLE With Discounting 


Straight Percent Discounting Method. 
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Figure 4.9 MLE With Discounting 
Estimate Variability at Low Constant Reltability. 
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Figure 4.10 MLE With Discounting 
Estimate Variability With Increasing Reliability. 
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Figure 4.11 Exponential Regression Model 
Lloyd Discounting Method. 
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Figure 4.12 Exponential Regression Model 
Straight Percent Discounting Method. 
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Figure 4.13 Exponential Regression Model 
Estimate Variability at Low Constant Reliability. 
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Figure 4.14 Exponential Regression Model 
Estimate Variabilitv With Increasing Reliability. 
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V. SUMMARY AND CONCLUSIONS 


A. SUMMARY 

The purpose of this study is to develop and evaluate discrete reliability growth 
models that employ fractional failure reduction, referred to as failure discounting. 
These models use all available data to estimate current reliability even if the actual 
reliability underlying all test data is not constant. This reduces the resources required 
to verify system reliability. This study focuses on developmental testing scenarios where 
testing is terminated after a fixed number of failures have been observed and the results 
of each trial are recorded as a success or failure, 1.e. attribute data. 

Failure discounting is performed on past data to make it compatible with test 
data taken from the current system configuration. Most reliability estimators require a 
constant actual reliability for each trial to render an accurate estimate. Fractionally 
discounting past failures adjusts test data from previous testing on less reliable system 
configurations making the data useful in the computation of the current reliability 
estimate. 

Two different failure discounting methods were used in. the study. The first, 
referred to as the Straight Percent Discounting Method allows the user to predetermine 
the fraction by which each failure will be reduced, the discount fraction, and an interval 
between applications of the discount fraction, the discount interval. The discount 
interval is the number of trials without a reoccurrence of the particular failure cause 
between application of the discount fraction. The second failure discounting method, 
referred to as the Lloyd Discounting Method, sets all past failures equal to the upper 
confidence bound for the probability of reoccurrence for the failure cause. Both failure 
_ discounting methods were used with each of the three discrete reliability growth models 
developed in the study. 

Three discrete reliability growth models using failure discounting were developed. 
The first model adds failure discounting to the maximum likelihood estimate for a 
proportion. The model is referred to as the Maximum Likelihood Estimate With 
Discounting Model. To use the model, the failure discounting method of choice is 
applied to the test data. The resulting discounted failures are returned to unity by 


adjusting the number of successful trials preceding each failure. The estimate of system 
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reliability is computed as the ratio of adjusted successful trials to adjusted total trials. 
Failure discounting allows the model to track changing reliability. 
The remaining two models are modifications of an exponential single phase 


reliability estimate of the form reliability = 1-4 


. One model uses regression to 
estimate A. This model is referred to as the Exponential Regression Model. The second 
model uses an unbiased estimate of A to produce a single testing phase reliability 
estimate. The current reliability estimate is a weighted average of the previous testing 
phase reliability estimate and the current single phase reliability estimate. The fraction 
of the total test trials to date used in the computation of each estimate becomes the 
weighting factor for the arithematic average. As with the Maximum Liklihood With 
Discounting Model, a failure discounting method is applied to the test data before 
either model is used to compute a reliability estimate. Both these models have the 
ability to track changing reliability without the use of failure discounting; but, failure 
discounting is employed to enhance the models performance. 

A Monte-Carlo simulation was written to evaluate the performance of the 
reliability growth models and failure discounting methods developed in the study. This 
was necessary because closed form solutions for the statistical properties of the models 
are not mathematically tractable. The simulation uses input parameters to generate a 
reliability growth pattern. Test data including the number of trials up to and including 
system failure and the cause of system failure are generated from the established 
reliability growth pattern. Each of the three reliability growth are applied to the 
simulated test data resulting in an estimate of system reliability at each testing phase. 
After this procedure is replicated a suitable number of times, estimates for the mean 
and standard deviation of each estimate are computed. These estimates are used to 
evaluate the performance ofthe reliability growth models. By varying the reliability 
growth pattern and the failure discounting parameters, the models performance over a 
wide range of situations may be evaluated. The conclusions reached from this analysis 


are included tn the following section. 


B. CONCLUSIONS 

Even though the analysis of the reliability growth models performed in the study 
is not comprehensive enough to state conclusions with certainty, several general 
conclusions are apparent. 

The Lloyd Failure Discounting Method should not be used with the three 
reliability growth models developed in the study. With few exceptions, this method 


64 


caused the growth models to overestimate system reliability. This tendency to 
overestimate was particularly evident at low reliabilities when little or no actual 
reliability growth occurred. The few situations where the method performed well, rapid 
reliability growth, were handled equally well by the Straight Percent Discounting 
Method. The Straight Percent Discounting Method’s flexibility makes the preferred of 
the two methods for fractionally discounting failures. 

The Weighted Average Reliability Growth Model, in its current form, should not 
be used to estimate system reliability. The model grossly overestimated reliability for 
most growth patterns tested. Even when no failure discounting was applied, the model 
overestimated svstem reliability. Adding either failure discounting method only 
increased the tendency to overestimate. 

Both the Maximum Liklihood Estimate With Discounting and the Exponential 
Regression Model exhibited performance superior to the standard, single phase MLE, 
estimate. But there is no clear cut choice between the Maximum Likelihood Estimate 
With Failure Discounting Model and the Exponential Regression Model. Both models 
accurately track a-wide range of reliability growth patterns. The MLE With 
Discounting Model appears to be the less variable of the two methods but is much 
more sensitive to the choice of failure discounting parameters than is the Exponential 
Regression Model. The variability of both models decreased with increasing reliability 
and additional test data. -For this reason the Exponential Regression Model is 
recommended for situations where four or more testing phases are planned and the 
system possesses high reliability with near certainty. For other cases the Maximum 
Likelihood Estimate With Failure Discounting is recommended because of its reduced 
variability. 

The discrete reliability growth models using failure discounting developed in the 
study represent an improvement over conventional reliability estimation techniques for 
the special case of testing until a fixed number of failures have been observed with 
attribute data. However, these models can be improved. The following section 


contains recomendations for improvements and further study. 


C. RECOMENDATIONS 
The following are recomendations for improvement to the models introduced in 
this study. 


e Many more simulation runs with varied reliability growth patterns and failure 
discounting parameters are required before solid conclusions concerning the 
properties of the reliability growth models can be reached. 
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The Monte-Carlo Simulation should be modified so declining reliability can be 
modeled. 


An alternate weighting technique should be developed for the Weighted 
Average Model to correct its tendency to overestimate reliability. In certain 
Situations, this model demonstrated the smallest variability of the three models 
developed. The model might prove to be an excellent reliability estimator if the 
overestimation tendency can be corrected. 


An additional failure discounting method that combines the flexibility of the 
Straight Percent Discounting Method with the non-arbitrary selection of the 
discounting fraction from the Lloyd method might prove useful. This could be 
accomplished by adding a discount interval to the current Lloyd Discounting 
Method. 


The possibility of using a simulation similar to the one developed here to aid in 
the selection of failure discounting parameters should be explored. Using an 
estimate of initial reliability, a reliability goal and the number of planned trials; 
the various choices for failure discounting parameters could be tested and one 
set chosen based on the simulation results. 


Further analysis of the performance of the reliability growth models with 
varying failure discounting parameters is required to develop “rules of thumb” 
for failure parameter selection. 
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APPENDIX A 


DISCRETE RELIABILITY GROWTH MODEL SIMULATION 


FORTRAN CODE FOR THE SIMULATION 


REKEKKKRKREKKRKRERRARRKRRRKRRRRRRRKRKRKRKRRRRRRRRRRRKRKRRRKRKRRRRRERKERERRRKRERRARRRKKRE 


DISCRETE RELIABILITY GROWTH SIMULATION 
PROGRAMMED BY JAMES E DRAKE 
LAST MODIFIED 19 AUG 1987 


THE FOLLOWING EXTERNAL FILES ARE USED BY THE PROGRAM 


INPUT 
THESIS 


: DATA AND PARAMETER INPUT FILE (DEVICE # 10) 
: OUTPUT FILE CONTAINING INTERMEDIATE COMPUTATIONS 


(DEVICE # 20) 
RELIAB: OUTPUT FILE CONTAINING FINAL RESULTS OF THE SIMULATION 
(DEVICE + 30) 


mol 


MLEWD 


MLESP 


REGEST 


ecu leU Peetes CONTAINING EACH PHASE ESTIMATE FOR EACH 


REPLICATION OF THE WEIGHTED AVERAGE ESTIMATE 
(DEVICE # 40) 


; OUTPUT FILE CONTAINING MLE ESTIMATES USING DISCOUNTING 


FOR EACH PHASE AND EACH REPLICATION 
(DEVICE # 50) 


; OUTPUT FILE CONTAINING MLE ESTIMATE FOR EACH SINGLE PHASE 


AND ALL REPLICATIONS USING NO DISCOUNTING 
(DEVICE # 60) 


: OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 


REPLICATION OF THE EXPONENTIAL REGRESSION ESTIMATE 
(DEVICE # 70) 


THE FOLLOWING IS A LIST OF KEY ARRAYS USED IN THE SIMULATION 


A 


NFAPH 
NFCAUS 


NTRIAL 


PHREST 


AREL 
YJIK 
CUMSF 


: MAIN WORKING ARRAY CONTAINS PROBABILITY OF SUCCESS FOR 


EACH FAILURE CAUSE, NUMBER OF TRIALS UNTIL FAILURE FOR 
EACH FAILURE CAUSE AND THE SYSTEM, CAUSE OF FAILURE, 
PHASE NUMBER, ADJUSTED NUMBER OF TRIALS AND ADJUSTED 
NUMBER OF FAILURES 

DIMENSION ( ((2*#CAUSES)+7),#FAILURES ) 


; CONTAINS THE NUMBER OF FAILURES IN EACH PHASE 


DIMENSION (1,#PHASES) 


; BINARY ARRAY USED TO DETERMINE IF A FAILURE OCCURRED IN 


A PHASE 
DIMENSION ( 1,#FAILURE CAUSES) 


: CONTAINS THE NUMBER OF TRIALS SINCE LAST FAILURE OR 


DISCOUNTING FOR EACH FAILURE CAUSE 
DIMENSION ( 1,#FAILURE CAUSES 


) 
: RECORDS THE PHASE ESTIMATE FOR EACH ESTIMATOR WITHIN A 


SINGLE REPLICATION 

DIMENSION (4,#PHASES) 

ROW 1 : WEIGHTED AVERAGE ESTIMATE 

ROW 2 : MLE WITH DISCOUNTING 

ROW 3 : SINGLE PHASE MLE 

ROW 4 : EXPONENTIAL REGRESSION ESTIMATE 


: CONTAINS ACTUAL SYSTEM RELIABILITY IN EACH PHASE 


DIMENSION (1,#PHASES ) 


seCONTAINS YJK VALUES UP TO 1000 


DIMENSION (1,1000) 


: CONTAINS THE NUMBER OF SUCCESS AND FAILURES FOR EACH 


FAILURE CAUSE (USED WITH WEIGHTED AVERAGE EST.) 
DIMENSION (3,#FAILURE CAUSES) 

ROW 1 : NUMBER OF FAILURES 

Pewee : NUMBER OF SUCCESSES 
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ROW 3 : ADJUSTED NUMBER OF SUCCESSES 


: ARRAY USED TO COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE 


DIMENSION (5, DED, 
ROW 1 : K 


Za: 

3 : ¥Y BAR FOR THE PHASE 
ROW 4 : B HAT 

5 : A HAT 


THE REMAINING ARRAYS ARE USED TO COMPUTE THE MEAN AND VARIANCE 
OF EACH ESTIMATE AT EACH PHASE. THEY ALL HAVE THE SAME DIMENSIONS 
AND STRUCTURE 


Boe 


MLEWD 
MLESP 
REGEST 


(Tae ee es ee 


DIMENSION (4,#PHASES ) 

ROW 1 : RUNNING SUM OF ESTIMATES 

ROW 2 : RUNNING SUM OF SQUARED ESTIMATES 
ROW 3 : MEAN OF THE ESTIMATES 

ROW 4 : STANDARD DEVIATION OF THE ESTIMATES 


VALUES FOR THE WEIGHTED AVERAGE ESTIMATE 


: VALUES FOR THE MLE WITH DISCOUNTING 
: VALUES FOR THE siliGhi aos en 
: VALUES FOR THE EXPONENTIAL REGRESSION ESTIMATE 


DEFINE AND DIMENSION VARIABLES 


PARAMETER (NR=50,NC=200) 
INTEGER REP, CUMSF, DIS@PT 
REAL*4 MIN 

REAL*8 DSEED,MLESP ,MLEWD,ES 


DIMENSION NFAPH(NR),A(NR, NC) ECSU (a pert ie oe NR 
CT(4,NR), Ae NR 
CMSF(3,NR),REG 


/MLESP (4 ,NR) ,REGEST(4,NR) ,AREL(NR) , YJK(1000 


3) 5 


READ IN THE NUMBER OF CAUSES TO BE USED ( NCAUSE ) AND THE NUMBER 
OF PHASES ( NPHASE ) IN THE TEST 


READ 
READ 


| 


167 zs NCAUSE 
10,*) NPHASE 


CREATE VARIABLES FOR THE ROW INDICES OF THE WORKING MATRIX ( A ) 
IPHASE: PHASE 

ISYSPReeaClLUALY COME ONE Niger in eng 

INTR: NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 

IFAILC: CAUSE OF THE FAILURE 


IADJF: 


ADJUSTED NUMBER OF FAILURES 
AFTER DISCOUNTING HAS BEEN APPLIED 


}; 


> SD SP Sa Sa SI SI OE SI SID Sa Sa SI, GO Sa SO, I SO, I SM 


,E9 
CU 


ED 


IADJT: ADJUSTED NUMBER OF TRIALS AFTER DISCOUNTING HAS BEEN APPLIED 
IYJK: YJK COMPUTED ON THE ADJUSTED NUMBER OF TRIALS 


IPHASE = (2*NCAUSE)+1 
ESYSPRe= seas 
INTR. =o 1 Sb Rees 
IFAILC = INTR + l 


IADJF 


LEA Ree +o ei 


LADdT = TAbDIrer 1 


IYdn 


= ITADJT + 1 


READ IN THE NUMBER OF FAILURES IN EACH PHASE ( NFAPH(I) ) AND 
COMPUTE THE TOTAL NUMBER OF FAILURES IN THE TEST ( NFAIL ) 


NFAIL = 0 
DO 10 I=1,NPHASE 


10 


READ (10, *) NFAPH(I) 
NFAIL = NFAIL + NFAPH(I) 


CONTINUE 


INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRisierOk BAeh CAUSE 
IN THE FIRST PHASE 


68 


DO 20 - He NCAUS 
ADCO, a Pilla) 
CONTINUE 


INPUT THE REMAINING VARIABL 
BEFORE A DISCOUNT IS APPLIE 


20 


GROWTH FRACTION (FRIMP); TR 

(IOPT) 

TRIGGERS FOR SAVING EACH ES 
TOPT1 : WEIGHTED AVERAGE 
BORTZ 7 MERGWITH DISCOUN 
B@ELS =]: OLNGLE PHASE MLE 
IOPT4 : EXPONENTIAL REGR 

DISCOUNTING OPTION TRIGGER 
PARAMETER (GAMMA) 


ANAANAANANANAN 


Sean 


ANREP = “NREP 
DSEEDIL DSEED 


mia lcALIZE THE ARRAYS USED 
SeeehCH ESTIMATOR 


De, s0 J=1, NPHASE 
DO 30 


C 
G 


30 CONTINUE 


FOR THE RANDOM NUMBER GENERATOR, GGUBFS, 


Ho, loo. NUMBER OF SUCEESSEUL TRIALS 

D (N); THE DISCOUNT FACTOR (R); THE SEED 
(DSEED) ; RELIABILITY 

IGGER FOR PRINTING INTERMEDIATE OUTPUT 


TIMATE AT EACH PHASE FOR EACH ESTIMATOR 
MODEL 
TING 


ESSION MODEL 
(DISOPT); LLOYDS FAILURE DISCOUNTING 


TO COMPUTE THE MEAN AND STANDARD DEVIATION 


feectiP ULE AND STORE THE YJK VALUES UP TO 1000 


— 


mai) = 0.0 
ies 2 


DO 40 a 
care ne 
40 CONTINUE 
C COMPUTE AND STORE K BAR FOR 
SUM = 0.0 
DO 50 I=1,NPHASE 
UM 
PoG(l,1) = 
50 CONTINUE 
C MAJOR REPETITION OF THE SIM 
DO 500 REP=1,NREP 


YJIK(T) 


C 
@ 


INITIALIZE FAILURE CAUSE VE 
SOMUPUTE THE INITIAL SYSTEM 


Bee = a1. 
DO 60 I=1, NCAUSE | 


60 


Oya 


THE EXPONENTIAL REGRESSION MODEL 


ULATION LOOP 


CTOR (NFCAUS) AND (CUMSF) 
RELIABILITY 
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C INITIALIZE COLUMN (FAILURE # ) COUNTER FOR THE WORKING ARRAY (A) 
Joo 


C LOOP TO COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 
C AND THE CAUSE OF FAILURE FOR EACH FAILURE IN EACH PHASE 


DO 130 K=1,NPHASE 


C SKIP ACTUAL COMPONENT RELIABILITY COMPUTATION AFTER FIRST REP 
C AND FOR FIRST FAILURE 


TE (Re £0. ymGOnounS 
IF(REP.GT.1) GOTO 75 
ime) =) oe 


C COMPUTE NEW ACTUAL RELIABILITY FOR THE COMPONENT VIN PHASE K 
DO 70 I=!l NCA 


C INCREASE CAUSE PR(SUCCESS) IF IT CAUSED FAILURE IN THE PREVIOUS PHASE 
C COMPUTE NEXT PHASE RELIABILITY AND REINITIALIZE NFCAUS 


IF (NECAUS(I).EQ.1) THEN 
A(I,J) = A(I,(J-15) + (Qe - A(I,(J-1)))*FRIMP) 
ELSEIF (NFCAUS 1) Nest) 7 

Ai J) = ACR oo 

ELSE 


ENDIF 
ELIS REL*A(T, J) 
NFCAUS(I) = 
70 CONTINUE 
Jl =12 


ips 
TRIOT = 9020 


C COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE AND THE 
C CAUSE OF FALLURE FOR EACH PAL LURE Othe eae se 


DO 120 L=1,NFAPH(K) 
a REP .GT. 1) GOTO 90 
IIT CTO) - 1) GOTO 85 

DO 80 I=1, , NCAUSE 
A(I,J) = A(I,(J-1)) 


80 CONTINUE 

35 A(ISYSPR,J) = REL 
A(IPHASE,J) = 

90 MIN = 7. 


2E75 
DO 110 I=1,NCAUSE 
C ASSIGN # TRIALS FOR CAUSES WITH PR(SUCCESS) = 0 OR 1 
IF(A(I,J).GE.1.) THEN 
A((I+NCAUSE),J) = 7.2E75 
GOTO 100 


ELSEIF(A(I,J). =Q. 0.) THEN 
a( (T+NCAUSE) , a) Sil. 
GOTO 100 


ELSE 
ENDIF 
C CONVERT UNIFORM (0,1) RANDOM VARIABLE TO GEOMETRIC (# TRIALS UNTIL 
C FAILURE ) FOR EACH FAILURE CAUSE. RECORD THE MIN # TRIALS FOR THE 
C CAUSES AS THE SYSTEM # TRIALS UP TO AND INCLUDING FAILURE AND 
C RECORD THE FAILURE CAUSE 
Crees J) = IN! . +(LOG (GGUBFS (DSEED))/LOG(A(T, THR 
100 IF(A((I+NCAUSE),J).LE.MIN) T 
_AC(T#NCAUSE) , J) 
IMIN = 
ELSE 
ENDIF 
110 CONTINUE 
A(IFAILC,J) = TMIN 
NFCAUS(IMIN) = 


C COMPUTE THE TOTAL # OF Fee FOR THE MLE SINGLE PRSSb estimate sane 
C INCREMENT FAILURE # COUNTERS 


710 


A(INTR,J) = MIN 
TRIOT = mater + A(INTR,J) 
dS ai 
da = Ji + 1 
120 CONTINUE 


feeeure ule THE MEE ESTIMATE OF COMPONENT RELIABILITY FOR THIS PHASE AND 
C COMPUTE THE RUNNING SUM OF ESTIMATES AND THE SUM OF ESTIMATES SQUARED 
C FOR COMPUTATION OF THE MEAN AND STANDARD DEVIATION OF THE ESTIMATE 


PHREST(3,K) = (IRTOT - BRE) TRIOL 
EEE ee K) = MLESE(1, i. Peer 0(s ,K) 
Mibor (2,6) — MLESP(2Z,K) + (PHREST (3, K)**2) 


130 CONTINUE 


PeelbileaLidcdeH IHE ADJUSTED NUMBER OF FAILURES TO 1 AND THE COUNT OF THE 
C NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING (NTRIALS(I) ) TO 0 
SeetierenEPARATION FOR THE DISCOUNTING ROUTINE 


DO 140 J=1,NFAIL 
BCIADIF,3) = 1. 
140 CONTINUE 


DO 150 I= a NCAUSE 
NT Pani) = 0 
150 CONTINUE 


C DISCOUNTING ROUTINE REVIEWS ALL PAST FAILURES AND CAUSES TO DATE 
Seo OeDA LE RMINES IF THE DISCOUNTING CONDITIONS HAVE BEEN MET. COMPUTES 
C THE ADJUSTED FAILURES, THE ADJUSTED w OF TRIALS AND YJK 


DO 300 K=1,NPHASE 
DO 200 L=1,NFAPH(R) 
rt ae ae 
C UPDATES THE NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING FOR EACH 
BeerelLURE CAUSE 


ICAUSE = BNE MGERILE, ota ) 
DO 160 I=1,NCAUS 
IF(ICAUSE. EQ. 1) THEN 


NTRIAL(I) = 0 
ELSEIF(ICAUSE.NE.I) THEN 
NERtaLon) = NT RIAL(D) + INT(A(INTR,J)+.5) 
ELSE 
ENDIF 
160 CONTINUE 


evo) CONTINUE 
©) CHOOSE DISCOUNTING METHOD TO BE USED 
IF(DISOPT.NE.2) GOTO 180 
C PERFORM LLOYDS FAILURE DISCOUNTING METHOD 


DO 170 I=1,J 
Il = “INT(A(IFAILC, ao) 
BEER Ee D) : ee 0) ae 
A(CIAD 18 


GOTO 176 
£LSE 
ENDIF 


A(IADJF,I) = 1.0 - 1.-GAMA)**(1.0/NTRIAL(I1 
170 CONTINUE yan vou 
GOTO 210 


C PERFORMS STRAIGHT PERCENT FAILURE DISCOUNTING AND 
= COMPUTES THE ADJUSTED + OF FAILURES 


180 DO 190 I=1,J 
Il = INT(A(IFAILC,I)+.5) 
IF (NTRIAL (11) .E .0) THEN 
A(IADJF,I 
ELSEIF (NTRIAL(I1).GE.N) THEN 
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_ A(TADSF, I) = A(IADJF,I)*((1.-R)**(NTRIAL(I1)/N) ) 


ENDIF 
190 CONTINUE 


C ADJUSTS THE # TRIALS SINCE FAILURE OR DISCOUNTING FOR THOSE CAUSES 
C THAT HAVE MET OR SURPASSED THE DISCOUNTING THRESHOLD 
C FOR THE STRAIGHT PERCENT DISCOUNTING METHOD 


DO 205 I=1,NCAUSE 
IF(NTRIAL(I). GE.N) NTRIAL(I) = MOD(NTRIAL(I) ,N) 
Z05 CONTINUE 
210 DAD Iie = 70). 0 
ey cree 
TeYIK = 0.0 
Kl = 0 


DOS 21971 2=i35 
DO y215_i=17 NCAUSE 
CUMSF(I2, I) =0 
LAWS CONTINUE 


C COMPUTES THE ADJUSTED # OF TRIALS FROM THE ADJUSTED # OF FAILURES 
C AND COMPUTES THE SUM OF THE ADJUSTED # OF TRIALS FOR ESTIMATE COMP. 


PREL = 0.0 
LTRIAL = 0 
I 


DO 240 I=1,J 


ATARI = -_ACINTR Zi /ACaMDa Re I) 
TADIT = TADIT + A(IADJT,1) 


COMPUTE YJK FROM THE ADJUSTED # OF TRIALS AND STORE THE SUM FOR 
Eola re TOG OME USE ARRAY FOR # TRIALS < 1000 AND APPROX. FOR 
VALUES > 1 


N1 = NINT(A(IADJT,I)) 
TF (N1.LE,1000 THEN 
A(IYJK,I) = YJK(N1) 
_ ELSEIF(N1.GT. :1000) THEN 
; A(IYIK,1) = (A(IADJT,1I)**.10995)*3.55445 


ENDIF 


C DETERMINE IF A PHASE BOUNDARY HAS BEEN REACHED TO BEGIN ESTIMATE 
C COMPUTATION 


sate ,EQnd i comomecs 
IF(A(IPHASE,I).NE.A(IPHASE,(I-1))) THEN 


C COMPUTE THE WEIGHTED AVERAGE ESTIMATE 


MAA = 0 
Kl =. Ki 


C DETERMINE THE FAILURE CAUSE WITH THE LARGEST # OF FAILURES 


DO 220 I1=1,NCAUSE 
IF(CUMSF(1,I1).GT.MAX) THEN 
MAX = CUMSF(1,11) 

ICOL = I1 
ELSE 


ENDIF 
220 CONTINUE 
C COMPUTE YJK VALUE FOR THE CURRENT PHASE es liiate 


TF (CUMSE(L, PCOn uae 1000) THEN 
AHATL = YJK(CUMSF(1,ICOL)) 
ELSEIF (CUMSF (1, TCOL). ene 1000) THEN 

X = CUMSF(1, ICOL) 

AHATL = CREE, 10995)*3.55445 
ELSE 
ENDIF 
IX = CUMSF(1,ICOL) + CUMSF(3,ICOL) 
IF(IX.LE.1000) THEN 


AADN 


a 


AHATU = YJK(IX) 
ELSEIF(IX. GT.1000) THEN 
xX = 


IX 
AHATU = (X**.10995)*3.55445 


Meer e Ure CURRENT PHASE RELIABILITY ESTIMATE 


AHAT = AHATU - AHATL 
CREL = 1.0 - EXP(-AHAT) 
me CUMSF(1,ICOL) + CUMSF(3,ICOL) 


C COMPUTE AND STORE THE WEIGHTED AVERAGE ESTIMATE 


PREL = Wee entice + (((X=LTRIAL)*CREL)/X) 
eeroNL = CUMSFE(i,ICOL) * CUMSP(3,ICOL) 


COMPUTE THE PHASE AND GLOBAL AVERAGE FOR YJK USED IN THE WOODS 
REGRESSION ESTIMATES ARE 


aa 


ee ai = Se ane 
REG(3,K1) = TPYJK/NFAPH(K1) 
BEYIK = 0.0 


ENDIF 


MeeolrUlE THE NUMBER OF FAILURES AND SUCCESSES FOR EACH FAILURE CAUSE 
Soc IN THE WEIGHTED AVERAGE ESTIMATE 


225 ICAUSE = INT(A(IFAILC,I)+.3) 
DO 230 I1=1,NCAUSE 
et eed = Bah Ea + INT(A(INTR,I) + .5) 


GUS: (Sytem CUMSE(3,11) + N1 
230 CONTINUE 
eUror (©, [CAUSE) = CUMSF(1,ICAUSE) + 1 
Oise 2, [CAUSE == CUMSF(Z,ICAUSE) - l 
mse (3, [CAUSE J) = CUMSEF(3,ICAUSE) = 1 








TPYJK = TPYJK + A(IYJK,I) 
TYJK = TYJK + A(IYIK,15 
240 CONTINUE 


C REPEAT COMPUTATIONS FOR THE WEIGHTED AVERAGE ESTIMATE FOR THE 
C FINAL PHASE 


MAX = 0 
Ki = Ki + 1 
DO 245 I1=1,NCAUSE 


IF(CUMSF(1,I1).GT.MAX) THEN 
MAX = cunSE (1, al 


ENDIF 
245 CONTINUE 


IF(CUMSF(1, IO LE. coe a 
AHATL = CUMSF(1,1 L)) 
ELSEIF (CUMSF(1, EOE). cn 1000) THEN 

x = CUMSE (1 ICOL ) 
AHATL = (XR , 10995)*3.55445 


ENDIF 
IX = CUMSF(1, 1 Cone me CUMSE( 3, FGCL ) 
inst COD TS OIGIC )) ese 
AHATU = vik (Tx x) 
eee STA. .GT.1000) THEN 


Ix 
AHATU = (X**,10995)%*3.55445 


AHAT = AHATU - AHATL 
Speigy= 1.0 = EXP 


(-AHAT ) 
x = CUMSF(1, ICOL) + CUMSF(3,ICOL) 
PREL = ((LTRIAL*PREL)/X) + (((X-LTRIAL)*CREL)/X) 


qe 


LTRIAL = CUMSE(1, TCOL) + Culisa Gs 1coL) 
Se = are 
REG(3,K1) = TPYJK/NFAPH(K1) 


PHREST(1,K) = PREL 

C COMPUTE THE MLE ESTIMATE OF PHASE RELIABILITY USING DISCOUNTING 
PHREST(2,K) = (TADJT - J)/TADJT 

C COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE BEGINNING WITH B HAT 


SUM = 0. 30, 

SUMS = 

IF (K. “9.1 2), GOTO 252 
DO 


SUM = Stn + ((I-REG(1, eee 
SUMS = SUMS + ((I-REG(1,R) )** 
250 CONTINUE 


REG(4,K) = SUM/SUMS 
C COMPUTE A HAT 
REG(5,K) = REG(2,K) - (REG(4,K)*REG(1,K)) 
C COMPUTE AND STORE THE EXPONENTIAL REGRESSION ESTIMATE 


PHREST(4,K) = 1.0 - EXP(- ae Be + (REG(4,K)%*K))) 

IF (PHREST (4, .K).LT.0.0) PHREST(4 Kijeno 

252 PHREST(4,K) = 1.0 - EXP(-REG(3,1)) 
IF(PHREST(4,X).LT.0.0) PHREST(4,K)=0.0 


C STORE THE RUNNING SUM OF THE ESTIMATES FOR THE CURRENT PHASE AND THE 
C RUNNING SUM OF THE ESTIMATES SQUARED FOR COMPUTATION OF THE MEAN AND 
C STANDARD DEVIATION OF EACH ESTIMATE FOR EACH RELIABILITY GROWTH 
255 EST(1, Ko] ESnar K) + PHREST(1,K) 
EST(2.K) = EST(2.K) + (PHREST(1,K)**2) 

nee = MLEWD (1, Ki + PHREST(2,K) 

MLEWD(2,K) = MLEWD(2,K) + (PHREST(2,K)**2) 

ee = ceo K} + PHREST(4,k) 

REGEST(2,K) = REGEST(2,K) + (PHREST(4,K)**2) 


C STORE THE ACTUAL PHASE RELIABILITY 
AREL(K) = A(ISYSPR,J) 


C PRINT INTERMEDIATE OUTPUT IF REQUESTED AND THE NUMBER OF REPETITIONS 
C IS NOT GREATER THAN 5 


PE oe .NE.1) GOTO 300 
EF( REP .Gr.5) GOLGRe 40 


WRITE(20,1000) REP,K 

1000 FORMAT(T16,'REPETITION NUMBER: ',14, PHASE NUMBER: ',I4) 
WRITE(20,1010) A(ISYSPR,J) 

1010 FORMAT(22X,'ACTUAL COMPONENT RELIABILITY: ',F7.5) 
WRITE(20,1020) PHREST(1,K 

1020 FORMAT(20X,'PREDICTED COMPONENT RELIABILITY: ',F7.5) 
WRITE (20,1022) PHREST(2,K 

1022 FORMAT(20X,'MLE ESTIMATE USING DISCOUNTING: ',F7.5) 

~  WRITE(20,1025) PHREST (3, K) 

1025 FORMAT(18X,'MLE ESTIMATE OF PHASE RELIABILITY: ',F7.5) 
WRITE (20,1027) PHREST(4,K) 

1027 FORMAT(14X,'REGRESSION ESTIMATE OF PHASE RELIABILITY: ',F7.5) 
WRITE (20,1030) 

1030 FORMAT 
DO 260 I=1,NCAU 

WRITE(20, 1035)T, A(T, J) ,A( (I+NCAUSE) , J) 

1035 op FORMAT (12x, ‘CAUSE: ',I3, PR(SUCCESS): ',F7.6,' # TRIALS: ', 

260 CONTINUE 
WRITE (20,1036) 

1036 FORMAT 
WRITE 20, 1040) 
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1040 FORMAT(4X,'FAIL #' 3X, ' FAIL CRUSE oe ff LRIADS Sk ADI f FAIL' ,3 
CX,'ADJ # ‘ae le ie aN ) 
DO 270 I=l1, 
WRITE(20, 1630)r, ALUM iee seat eNtR, () ,ACLADJF,1),A(IADJT,1),A(IYJK 


Cal 
1050 FORMAT (4X,13,8X,F3.0,7X,F8.0,4X, F8.6,4X,F12.0,3X,F11.4) 
270 CONTINUE 
Beret 1060} 
1060 FORMAT ry 


300 CONTINUE 
Meee BACH OF THE 3 ESTIMATES TO THEIR APPROPRIATE OUTPUT FILE 
See oe REOUESTED 


IF(IOPT1.NE.1) GOTO 401 
400 WRITE(40, 2000} (PHREST(1,I), I=1,NPHASE) 
401 IF(IOPT2.NE.1) GOTO 402 

WRITE (50, 2000) BES E) I), I=1,NPHASE) 
402 IF(IOPT3. NE. ik) G 40 

WRITE (60, 2000) (PHREST (3, I), I=1,NPHASE) 
403 IF(IOPT4. NE.1) GOTO 500 

WRITE (70 2000) (SHREST (4, I), I=1,NPHASE) 
Z000 FORMAT POO Ch dae > 1 x))) 


200 CONTINUE 


C UFON COMPLETION OF ALL REPETITIONS, COMPUTE THE MEAN AND STANDARD 
Pee ay eal tON OF EACH ESTIMATE FOR EACH PHASE SKIPPING COMPUTATIONS IF 
Seeevcy ONE REPETITION IS REQUIRED 


Poe Ree .LE.1) GOTO 601 


DO 600 I=1, NPHASE | 
EST(3,I) = ES ST (1, ) /XNREP 
eee} 3 2) = ML sl CRS es cet 
MLESP(3,I) = MLESP(1,1)/XNREP 
ee = Pg epee REP 

I EST (XNREP*(EST(3,1)**2)))/(XNREP-1) 
HEEND(4,1) = = SORT _ 2,1)-(XNREP* aie ie weed / dibep. 4 
MLESP(4,1) = SORT( (MLESP(2,1)-(XNREP*(MLESP ee Le XNREP- i 
REGEST (4,1) =SORT( (REGEST(2,1)-(XNREP*(REGEST (3,1) **2 (XNREP-1 











eee e iNT THE aes OUTPUT TABLE TO A FILE 


601 WRITE(30,3000) 
3000 FORMAT('0',T47,'DISCRETE RELIABILITY GROWTH SIMULATION’ ) 
WRITE (30,3010 
3010 FORMAT('-',T54 > 'MODEL PARAMETER SUMMARY’ ) 
WRITE(30,3020) NCAUSE 
3020 FORMAT('0' ,T47,'NUMBER OF POSSIBLE FAILURE CAUSES ',14) 
WRITE(30, 3030 
3030 FORMAT('0',T38,'CAUSE NUMBER',1T64,'SINGLE TRIAL PR( SUCCESS ) FOR 
CPHASE 1') 
0 3050 M= NCAU USE 
WRITE (30, "3040)" M . “a 
3040 FORMAT /T43,12,T79,F8.6) 
3050 CONTINUE 
a FRIMP 
3060 eee FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 
WRITE (30,3080) NPHASE 
3080 FORMAT('-',T48,>'NUMBER OF PHASES IN THE SIMULATION ',1I2) 
WRITE (30,3090 
3090 eo", /T42,'PHASE NUMBER! ,T59,'NUMBER OF FAILURES IN THE FIRST 
DO 3110 M=1,NPHASE 
2°, ,3100) M,NFAPH(M) 
3100 FORMAT ,T43,12,T73,12) 
3110 CONTINUE 
WEI TE (30, ,3120) NFAIL 
3120 FORMAT('O',T51,'TOTAL NUMBER OF FAILURES ',14) 
IF(DISOPT. EQ. eGo TO 3150 
WRITE (30,3130) 


is 


3130 FORMAT('-!,T38,'DISCOUNTING PERFORMED USING THE CONSTANT FRACTION 
CMETHOD' ) 
WRITE(30,3140) R 
3140 FORMAT('0',T44,' FRACTION EACH FAILURE IS DISCOUNTED ',F8.6) 
WRITE (30, 31505 N 


3150 FORMAT(' ',T33,'NUMBER OF TRIALS AFTER A FAILURE BEFORE A DISCOUNT 
Cars APPLIED ' 14) 
GO7 Te, 3130 


S160 WRITE (30, 3170) 

3170 FORMAT ,T44, 'DISCOUNTING PERFORMED USING THE LLOYD METHOD' ) 
WRITE 30, 3180) GAMA 

3180 aes 'O' 139) ' PERCENT C.I. ( USED AS DISCOUNT FRACTION ) Sb ome 


2190 WRITE (30 , 3200) DSEED1L 


3200 FORMAT T46,'RANDOM NUMBER SEED USED ',F15.2) 
WRITE 30, 3210) NREP 
3210 FORMAT '0' 737,'NUMBER OF REPETITIONS OF THE SIMULATION PERFORMED 
WRITE (30,3220) 
3220 FORMAT('1' ,T61, ‘ESTIMATOR: ') 
WRITE (30,3230 
3230 FORMAT('0! ,T48,'SINGLE PHASE MLE WITHOUT DISCOUNTING' ) 
WRITE (30,3240 
3240 FORMAT('-' T60,'MEAN',T83, 'ESTIMATE' ,T109,'95 %') 
WRITE 30, 4520 
3250 FORMAT(' ',T12,'PHASE NUMBER! ,T29,'ACTUAL RELIABILITY! ,T52,'PREDIC 
c7ED Be on fee a ,T78, ‘STANDARD DEVIATION! ,T101,'CONFIDENCE INTERVA 


C COMPUTE Cl eanen tem PHASE MLE 


DO 3270 M=1 ,NPHAS 
in SeAILESE (4, M) ) /SQRT (XNREP ) 
aac = MLESP(3,M 
CIL = = Hees M) - 
WRITE (30,32 0§ M, AREL (I), MLESP(3,M),MLESP(4,M) ,CIL,CIU 
3260 FORMAT coe 'T34,F8.6,T58,F8.6,T82,F9.6,T99,'( ',F8.6,' , ',F 


3270 CONTINUE 
Were 50s s220 
WRETE( 30,6250 

3250 oa 'Q',T42,'MAX LIKELIHOOD ESTIMATE USING DISCOUNTED FAILURES 


WRITE (30,3240 
WRITE( 30,3250 


€ COMPUTE Col. fen Mee pos DISCOUNTING 


DO 3290 M=1,NPHA 
= (1. SO*ULEWD (4, i) ) /SQRT (XNREP ) 

ctu = MLEWD(3,M) + 
CIL = MLEWD(3,M 
WRITE (30,3260) f AREL(M) , MLEWD(3,M) ,MLEWD(4,M) ,CIL, CIU 

3290 CONTINUE 
WRITE (30,3220 
WRITE(30, 3300 

3300 FORMAT '0' 738, 'WEIGHTED AVERAGE ESTIMATE USING FAILURE DISCOUNTIN 


G 
ee eet 
WRITE( SO e250 


C COMPUTE C.I. FOR WEIGHTED AVERAGE ESTIMATES 


DO 3310 M=1 ,NPHASE 
(aie S6*EST(4, #)) /SQRT(XNREP) 
clu = EST(3, i 
Cll = ners 
WRITE (30, 3360 
3310 CONTINUE 


WgtTE 30/3320 








CI 
M,AREL(M) , EST(3,M),EST(4,M) , CIL, CIU 


SICIU TA SOL ec 
3320 FORMAT('0!,T43,'REGRESSION ESTIMATE USING DISCOUNTED FAILURES' ) 
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penae ore oes 
WRITE (30,3250 


Seecolrure C.f. FOR EXPONENTIAL REGRESSION ESTIMATES 


630 
3340 
5550 


3360 


3370 


8375 


3380 
650 


DO 3330 M=1,NPHASE 

ci = 1a SeRREGEST (4, M)) /SQRT(XNREP) 

CIU = REGEST 3, M 

elle REGEST(3 éM Ct 

WRITE(30, 5360) 'M. AREL(M) ,REGEST(3,M) ,REGEST(4,M) ,CIL, CIU 
CONTINUE 

WRITE( 30,3340) 

FORMAT('1!',T59, 'RECAPITULATION'//) 

WRITE( 30, , 3350 

FORMAT ,T3,'PHASE' ,T11, 'ACTUAL' ,T28, 'MEAN' ,T38,'EST! ,T53, 'MEAN' 
Mieco ect) fre) iNGaN® madmanen eioS, (MEAN 2113 (ner J 


S5a0), 
' 1 T11,'RELIAB' ,T28,'WGT',T38,'STD! ,T53,'MLE' ,T63,'STD! ,T7 
BopeCHASE' 788, 'STD' ,T103 "REG! ,T113,'STD') 
eT 3370 
| 1 °728,'AVG! ,T35,'!DEVIATION' ,T53,'W/D! ,T60, ‘DEVIATION! ,T78 
C,'MLE',T85,'DEVIATION' ,T103,'EST!,T110, ‘DEVIATION’ ) 


WRITE(30, 3380) 1, AREL(1) Botts, sey t) MLEWD(3,I), aie Plo 
SMLESr (3,1), MLESP(4 ee REGEST(3., ie REGEST 4, 


FORMAT('0O' ,T4,13, T11,27.6 wuz’, PS TS6, a. 6, 731, EoeGylol, Sc, 
o57.0,186, 7 cme E or ‘Tiil, et, 6) 

CONTINUE 

SLOP 

END 
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2. SAMPLE SIMULATION INPUT FILE 


5 NUMBER OF FPAILUREI@AU SEs 

10 NUMBER OF PHASES ( NPHASE ) 

l NUMBER OF FATLURES IN PHASE 1 

l NUMBER OF FAILURES IN PHASE 2 

NUMBER OF FAILURES IN PHASE 3 

l NUMBER OF FAILURES IN PHASE 4 

l NUMBER OP FAILURES IN PHASE 5 

l NUMBER OF FAILURES IN PHASE 6 

i NUMBER OP FAILURES IN PEASE 7 

l NUMBER OF FAILURES IN PHASE 8 

l NUMBER OF FAILURESSLN PHASE 

| NUMBER OF FAILURES IN Poaoe ay 

8 | - PROB. OF OCCURRENCE FOR CAUSE WIN PoAsem 
8 1 = PROB. OF OCCURRENCE POR CAUSE 2 alin ta 
8 1 - PROB. OF OCCURRENCE FOR CAUSE 3 IN PHASE 1 
i 1 - PROB. OF OCCURRENCE FOR CAUSE 4 IN PHASE 1 
a 1 - PROB. OF OCCURRENCE FOR GAUSE SIN PoC ed 
50 DISCOUNT INTERVAL FOR STRAIGHT PERCENT DISCOUNTING 


75 DISCOUNT FRACTION FOR STRAIGHT PERCENT DISCOUNTING 
624712.0 RANDOM NUMBER SEED 

95 RELIABILITY GROWTH FRACTION 

10000 NUMBER OF DESIRED REPETITIONS FOR THE SIMULATION 

0 INTERMEDIATE OUTPUT OPTION (1:INT. OUTPUT; 0: NO OUTPUT) 
0 SAVE ALL WEIGHTED AVERAGE ESTIMATES (1: YES; 0: NO ) 

0 SAVE ALL MLE W/ DISCOUNTING ESTIMATES (1: YES; 0: NO ) 

0 SAVE ALL MLE SINGLE PHASE ESTIMATES (1: YES; 0; NO ) 

0 SAVE ALL REGRESSION ESTIMATES (1: YES; 0: NO ) 

, 


DISCOUNTING OPTION (1: STRAIGHT % ; 2: LLOYD METHOD) 
GAMMA FOR LLOYD DISCOUNTING METHOD 


Nv 
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3. SAMPLE SIMULATION OUTPUT FILE 


OISCRETE RELIABILITY GROWTH SIMULATION 


MODEL PARAMETER SUMMARY 


NUMBER OF POSSIBLE FAILURE CAUSES 5 


CAUSE NUMBER SINGLE TRIAL PR( SUCCESS ) FOR PHASE 1 
-1 0.8000c0 
2 0.800000 
; 3 0.800000 
4 0.700000 
s ‘ 0.700000 


FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 0.950000 


NUMBER OF PHASES IN THE SIMULATION 10 


PHASE NUMBER NUMBER OF FAILURES IN THE FIRST PHASE 
1 
2 


Owen HW LD Ww 
OP OO Oo be et pet Oe pe 


“~ 


TOTAL NUMBER OF FAILURES 10 


OISCOUNTING PERFORMED USING THE CONSTANT FRACTION METHOO 
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APPENDIX B 
GRAPHICAL RESULTS OF THE RELIABILITY GROWTH MODEL SIMULATION 


Il MEAN ESTIMATES FOR EACH ESTIMATOR WITH CONSTANT PARAMETERS 
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LLOYD DISCOUNTING; GAMMA = .999 
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